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Most startups fail. But many of those failures 
are preventable. The Lean Startup is a new 
approach being adopted across the globe, 
changing the way companies are built and 
new products are launched. 


Eric Ries defines a startup as an organization 
dedicated to creating something new under condi- 
tions of extreme uncertainty. This is just as true 
for one person in a garage as it is in a group of 
seasoned professionals in a Fortune 500 board- 
room. What they all have in common is a mission 
to penetrate the fog of uncertainty to discover a 
successful path to a sustainable business. 

The Lean Startup approach fosters com- 
panies that are more capital efficient and that 
leverage human creativity more effectively. 
Inspired by lessons from lean manufacturing, 
it relies on “validated learning,” rapid scientific 
experimentation, as well as a number of coun- 
terintuitive practices that shorten product 
development cycles, measure actual progress 
without resorting to vanity metrics, and help 
us learn what customers really want. It enables 
a company to shift directions with agility, alter- 
ing plans inch by inch, minute by minute. 

Rather than wasting time creating elaborate 
business plans, The Lean Startup offers entrepre- 
neurs—in companies of all sizes—a way to test 
their vision continuously, to adapt and adjust 
before it’s too late. Ries provides a scientific 
approach to creating and managing success- 
ful startups in an age when companies need to 
innovate more than ever. 


‘Acclaim for THE LEAN STARTUP 


“The Lean Startup isn't just about how to create a more success- 
ful entrepreneurial business; it’s about what we can learn from 
those businesses to improve virtually everything we do. I imag- 
ine Lean Startup principles applied to government programs, 
° 5) > 
to health care, and to solving the world’s great problems. Its 
ultimately an answer to the question How can we learn more 
uickly what works and discard what doesn't?” 
q y 


—Tim O’Reilly, CEO, O’Reilly Media 


“Eric Ries unravels the mysteries of entrepreneurship and re- 
veals that magic and genius are not the necessary ingredients 
for success but instead proposes a scientific process that can be 
learned and replicated. Whether you are a startup entrepreneur 
or corporate entrepreneur, there are important lessons here for 
you on your quest toward the new and unknown.” 


—Tim Brown, CEO, IDEO 


“The road map for innovation for the twenty-first century. The 
ideas in The Lean Startup will help create the next industrial 
revolution.” 


—Steve Blank, lecturer, Stanford University, 
UC Berkeley Haas Business School 


“Every founding team should stop for forty-eight hours and 
read The Lean Startup. Seriously, stop and read this book now.” 


—Scott Case, CEO, Startup America Partnership 


“The key lesson of this book is that startups happen in the 
present—that messy place between the past and the future 
where nothing happens according to PowerPoint. Ries’s ‘read 
and react’ approach to this sport, his relentless focus on vali- 
dated learning, the never-ending anxiety of hovering between 
‘persevere’ and ‘pivot,’ all bear witness to his appreciation for the 
dynamics of entrepreneurship.” 


—Geoffrey Moore, author, Crossing the Chasm 


“If you are an entrepreneur, read this book. If you are thinking 
about becoming an entrepreneur, read this book. If you are just 
curious about entrepreneurship, read this book. Starting Lean is 
today’s best practice for innovators. Do yourself a favor and read 


this book.” 


—Randy Komisar, founding director of TiVo and 
author of the bestselling The Monk and the Riddle 


“How do you apply the fifty-year-old ideas of Lean to the fast- 
paced, high-uncertainty world of startups? This book provides 
a brilliant, well-documented, and practical answer. It is sure to 
become a management classic.” 


—Don Reinertsen, author, 
The Principles of Product Development Flow 


What would happen if businesses were built from the ground 

up to learn what their customers really wanted? The Lean Startup 

is the foundation for reimagining almost everything about how 

work works, Don't let the word startup in the title confuse you. 

This is a cookbook for entrepreneurs in organizations of all 
— 

sizes. 


—Roy Bahat, president, IGN Entertainment 


“The Lean Startup is a foundational must-read for founders, 
enabling them to reduce product failures by bringing structure 
and science to what is usually informal and an art. It provides 
actionable ways to avoid product-learning mistakes, rigorously 
evaluate early signals from the market through validated learn- 
ing, and decide whether to persevere or to pivot, all challenges 
that heighten the chance of entrepreneurial failure.” 


—Noam Wasserman, professor, Harvard Business School 


“One of the best and most insightful new books on entrepre- 
neurship and management I’ve ever read. Should be required 
reading not only for the entrepreneurs that I work with, but 
for my friends and colleagues in various industries who have 
inevitably grappled with many of the challenges that The Lean 
Startup addresses.” 


—Eugene J. Huang, partner, True North Venture Partners 


“Every entrepreneur responsible for innovation within their 
organization should read this book. It entertainingly and me- 
ticulously develops a rigorous science for the innovation process 
through the methodology of “lean thinking.” This methodology 
provides novel and powerful tools for companies to improve the 
speed and efficiency of their innovation processes through mini- 
mum viable products, validated learning, innovation account- 
ing, and actionable metrics. These tools will help organizations 
large and small to sustain innovation by effectively leveraging 
the time, passion, and skill of their talent pools.” 


—Andrea Goldsmith, professor of electrical engineering 
at Stanford University and cofounder of several startups 


“In business, a ‘lean’ enterprise is sustainable efficiency in ac- 
tion. Eric Ries’s revolutionary Lean Startup method will help 
bring your new business idea to an end result that is successful 
and sustainable. You'll find innovative steps and strategies for 
creating and managing your own startup while learning from 
the real-life successes and collapses of others. This book is a 
must-read for entrepreneurs who are truly ready to start some- 

thing great!” 
—Ken Blanchard, coauthor of The One Minute Manager® 
and The One Minute Entrepreneur 


“Business is too important to be left to luck. Eric reveals the rig- 
orous process that trumps luck in the invention of new products 
and new businesses. We’ve made this a centerpiece of how teams 
work in my company . . . it works! This book is the guided tour 
of the key innovative practices used inside Google, Toyota, and 
Facebook that work in any business.” 


—Scott Cook, founder and chairman of 
the Executive Committee, Intuit 
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Introduction 


top me if you ve heard this one before. Brilliant college kids sit- 

ting in a dorm are inventing the future. Heedless of bound- 
aries, possessed of new technology and youthful enthusiasm, 
they build a new company from scratch. Their early success al- 
lows them to raise money and bring an amazing new product to 
market. They hire their friends, assemble a superstar team, and 
dare the world to stop them. 

Ten years and several startups ago, iat was me, baits my 
first company. I particularly remember a moment from back 
then: the moment I realized my company was going to fail. My 
cofounder and I were at our wits’ end. The dot-com bubble had 
burst, and we had spent all our money. We tried desperately to 
raise more capital, and we could not. It was like a breakup scene 
from a Hollywood movie: it was raining, and we were arguing 
in the street. We couldn't even agree on where to walk next, and 
so we parted in anger, heading in opposite directions. As a meta- 
phor for our company’s failure, this image of the two of us, lost 
in the rain and drifting apart, is perfect. 

It remains a painful memory. The company limped along for 
months afterward, but our situation was hopeless. At the time, 
it had seemed we were doing everything right: we had a great 
product, a brilliant team, amazing technology, and the right 
idea at the right time. And we really were on to something. We 
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were building a way for college kids to create online profiles for 
the purpose of sharing... with employers. Oops. But despite 
a promising idea, we were nonetheless doomed from day one, 
because we did not know the process we would need to use to 
turn our product insights into a great company. 

If you've never experienced a failure like this, it is hard to de- 
scribe the feeling. It’s as if the world were falling out from under 
you. You realize you've been duped. The stories in the magazines 
are lies: hard work and perseverance don't lead to success. Even 
worse, the many, many, many promises you've made to employ- 
ees, friends, and family are not going to come true. Everyone 
who thought you were foolish for stepping out on your own will 
be proven right. 

It wasnt supposed to turn out that way. In magazines and 
newspapers, in blockbuster movies, and on countless blogs, we 
hear the mantra of the successful entrepreneurs: through de- 
termination, brilliance, great timing, and—above all—a great 
product, you too can achieve fame and fortune. 

There is a mythmaking industry hard at work to sell us that 
story, but I have come to believe that the story is false, the prod- 
uct of selection bias and after-the-fact rationalization. In fact, 
having worked with hundreds of entrepreneurs, I have seen 
firsthand how often a promising start leads to failure. The grim 
reality is that most startups fail. Most new products are not suc- 
cessful. Most new ventures do not live up to their potential. 

Yet the story of perseverance, creative genius, and hard work 
persists. Why is it so popular? I think there is something deeply 
appealing about this modern-day rags-to-riches story. It makes 
success seem inevitable if you just have the right stuff. It means 
that the mundane details, the boring stuff, the small individual 
choices don’t matter. If we build it, they will come. When we 
fail, as so many of us do, we have a ready-made excuse: we didn’t 
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have the right stuff. We weren't visionary enough or weren't in 
the right place at the right time. 

After more than ten years as an entrepreneur, I came to reject 
that line of thinking. I have learned from both my own successes 
and failures and those of many others that it’s the boring stuff 
that matters the most. Startup success is not a consequence of 
good genes or being in the right place at the right time. Startup 
success can be engineered by following the right process, which 
means it can be learned, which means it can be taught. 

Entrepreneurship is a kind of management. No, you didnt 
read that wrong. We have wildly divergent associations with these 
two words, entrepreneurship and management. Lately, it seems 
that one is cool, innovative, and exciting and the other is dull, 
serious, and bland. It is time to look past these preconceptions. 

Let me tell you a second startup story. It’s 2004, and a group 
of founders have just started a new company. Their previous 
company had failed very publicly. Their credibility is at an 
all-time low. They have a huge vision: to change the way people 
communicate by using a new technology called avatars (remem- 
ber, this was before James Cameron’s blockbuster movie). They 
are following a visionary named. Will Harvey, who paints a com- 
pelling picture: people connecting with their friends, hanging 
out online, using avatars to give them a combination of intimate 
connection and safe anonymity. Even better, instead of having 
to build all the clothing, furniture, and accessories these ava- 
tars would need to accessorize their digital lives, the customers 
would be enlisted to build those things and sell them to one 
another. 

The engineering challenge before them is immense: creat- 
ing virtual worlds, user-generated content, an online commerce 
engine, micropayments, and—last but not least—the three- 
dimensional avatar technology that can run on anyones PC. 


4 \ntroduction 


I’m in this second story, too. I’m a cofounder and chief tech- 
nology officer of this company, which is called IMVU. At this 
point in our careers, my cofounders and I are determined to 
make new mistakes. We do everything wrong: instead of spend- 
ing years perfecting our technology, we build a minimum vi- 
able product, an early product that is terrible, full of bugs and 
crash-your-computer-yes-really stability problems. Then we 
ship it to customers way before it’s ready. And we charge money 
for it. After securing initial customers, we change the product 
constantly—much too fast by traditional standards—shipping 
new versions of our product dozens of times every single day. 

We really did have customers in those early days—true vi- 
sionary early adopters—and we often talked to them and asked 
for their feedback. But we emphatically did not do what they 
said. We viewed their input as only one source of information 
about our product and overall vision. In fact, we were much 
more likely to run experiments on our customers than we were 
to cater to their whims. 

Traditional business thinking says that this approach 
shouldn't work, but it does, and you don’t have to take my word 
for it. As you'll see throughout this book, the approach we pi- 
oneered at IMVU has become the basis for a new movement 
of entrepreneurs around the world. It builds on many previous 
management and product development ideas, including lean 
manufacturing, design thinking, customer development, and 
agile development. It represents a new approach to creating con- 
tinuous innovation. It’s called the Lean Startup. 

Despite the volumes written on business strategy, the key at- 
tributes of business leaders, and ways to identify the next big 
thing, innovators still struggle to bring their ideas to life. This 
was the frustration that led us to try a radical new approach 
at IMVU, one characterized by an extremely fast cycle time, a 
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focus on what customers want (without asking them), and a sci- 
entific approach to making decisions. 


ORIGINS OF THE LEAN STARTUP 


I am one of those people who grew up programming comput- 
ers, and so my journey to thinking about entrepreneurship and 
management has taken a circuitous path. I have always worked 
on the product development side of my industry; my partners 
and bosses were managers or marketers, and my peers worked in 
engineering and operations. Throughout my career, I kept hav- 
ing the experience of working incredibly hard on products that 
ultimately failed in the marketplace. | 

At first, largely because of my background, I viewed these as 
technical problems that required technical solutions: better ar- 
chitecture, a better engineering process, better discipline, focus, 
or product vision. These supposed fixes led to still more failure. 
So I read everything I could get my hands on and was blessed 
to have had some of the top minds in Silicon Valley as my men- 
tors. By the time I became a cofounder of IMVU, I was hungry 
for new ideas about how to build a company. 

I was fortunate to have cofounders who were willing to ex- 
periment with new approaches. They were fed up—as I was—by 
the failure of traditional thinking. Also, we were lucky to have 
Steve Blank as an investor and adviser. Back in 2004, Steve had 
just begun preaching a new idea: the business and marketing 
functions of a startup should be considered as important as en- 
gineering and product development and therefore deserve an 
equally rigorous methodology to guide them. He called that 
methodology Customer Development, and it offered insight 
and guidance to my daily work as an entrepreneur. 
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Meanwhile, I was building IMVU’s product development 
team, using some of the unorthodox methods I mentioned ear- 
lier. Measured against the traditional theories of product devel- 
opment I had been trained on in my career, these methods did 
not make sense, yet I could see firsthand that they were working. 
I struggled to explain the practices to new employees, investors, 
and the founders of other companies. We lacked a common lan- 
guage for describing them and concrete principles for under- 
standing them. 

I began to search outside entrepreneurship for ideas that 
could help me make sense of my experience. I began to study 
other industries, especially manufacturing, from which most 
modern theories of management derive. I studied lean manu- 
facturing, a process that originated in Japan with the Toyota 
Production System, a completely new way of thinking about 
the manufacturing of physical goods. I found that by apply- 
ing ideas from lean manufacturing to my own entrepreneurial 
challenges—with a few tweaks and changes—I had the begin- 
nings of a framework for making sense of them. 

This line of thought evolved into the Lean Startup: the ap- 
plication of lean thinking to the process of innovation. 

IMVU became a tremendous success. IMVU customers have 
created more than 60 million avatars. It is a profitable company 
with annual revenues of more than $50 million in 2011, employ- 
ing more than a hundred people in our current offices in Moun- 
tain View, California. IMVU’s virtual goods catalog—which 
seemed so risky years ago—now has more than 6 million items 
in it; more than 7,000 are added every day, almost all created by 
customers. | 

As a result of IMVU’s success, I began to be asked for advice 
by other startups and venture capitalists. When I would describe 
my experiences at IMVU, I was often met with blank stares or 
extreme skepticism. The most common reply was “That could 
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never work!” My experience so flew in the face of conventional 
thinking that most people, even in the innovation hub of Sili- 
con Valley, could not wrap their minds around it. 

Then I started to write, first on a blog called Startup Les- 
sons Learned, and speak—at conferences and to companies, 
startups, and venture capitalists—to anyone who would listen. 
In the process of being called on to defend and explain my 
insights and with the collaboration of other writers, thinkers, 
and entrepreneurs, I had a chance to refine and develop the 
theory of the Lean Startup beyond its rudimentary beginnings. 
My hope all along was to find ways to eliminate the tremen- 
dous waste I saw all around me: startups that built products 
nobody wanted, new products pulled from the shelves, count- 
less dreams unrealized. 

Eventually, the Lean Startup idea blossomed into a global 
movement. Entrepreneurs began forming local in-person groups 
to discuss and apply Lean Startup ideas. There are now orga- 
nized communities of practice in more than a hundred cities 
around the world.’ My travels have taken me across countries 
and continents. Everywhere I have seen the signs of a new entre- 
preneurial renaissance. The Lean Startup movement is making 
entrepreneurship accessible to a whole new generation of found- 
ers who are hungry for new ideas about how to build successful 
companies. 

Although my background is in high-tech software entrepre- 
neurship, the movement has grown way beyond those roots. 
Thousands of entrepreneurs are putting Lean Startup principles 
to work in every conceivable industry. I’ve had the chance to 
work with entrepreneurs in companies of all sizes, in different 
industries, and even in government. This journey has taken me 
to places I never imagined I'd see, from the world’s most elite 
venture capitalists, to Fortune 500 boardrooms, to the Penta- 
gon. The most nervous I have ever been in a meeting was when 
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I was attempting to explain Lean Startup principles to the chief 
information officer of the U.S. Army, who is a three-star general 
(for the record, he was extremely open to new ideas, even from 
a civilian like me). 

Pretty soon I realized that it was time to focus on the Lean 
Startup movement full time. My mission: to improve the suc- 
cess rate of new innovative products worldwide. The result is the 
book you are reading. 


THE LEAN STARTUP METHOD 


This is a book for entrepreneurs and the people who hold them 
accountable. The five principles of the Lean Startup, which in- 
form all three parts of this book, are as follows: 


1, Entrepreneurs are everywhere. You don’t have to work 
in a garage to be in a startup. The concept of entrepreneurship 
includes anyone who works within my definition of a startup: a 
human institution designed to create new products and services 
under conditions of extreme uncertainty. That means entrepre- 
neurs are everywhere and the Lean Startup approach can work 
in any size company, even a very large enterprise, in any sector 
or industry. 


2. Entrepreneurship is management. A startup is an insti- 
tution, not just a product, and so it requires a new kind of man- 
agement specifically geared to its context of extreme uncertainty. 
In fact, as I will argue later, I believe “entrepreneur” should be 
considered a job title in all modern companies that depend on 
innovation for their future growth. 


3. Validated learning. Startups exist not just to make stuff, 
make money, or even serve customers. They exist to learn how 
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to build a sustainable business. This learning can be validated 
scientifically by running frequent experiments that allow entre- 
preneurs to test each element of their vision. 


4, Build-Measure-Learn. The fundamental activity of a 
startup is to turn ideas into products, measure how customers 
respond, and then learn whether to pivot or persevere. All suc- 
cessful startup processes should be geared to accelerate that feed- 
back loop. 


5. Innovation accounting. To improve entrepreneurial out- 
comes and hold innovators accountable, we need to focus on 
the boring stuff: how to measure progress, how to set up mile- 
stones, and how to prioritize work. This requires a new kind 
of accounting designed for startups—and the people who hold 
them accountable. 


Why Startups Fail 


Why are startups failing so badly everywhere we look? 

The first problem is the allure of a good plan, a solid strat- 
egy, and thorough market research. In earlier eras, these things 
were indicators of likely success. The overwhelming temptation 
is to apply them to startups too, but this doesn’t work, because 
startups operate with too much uncertainty. Startups do not yet 
know who their customer is or what their product should be. As 
the world becomes more uncertain, it gets harder and harder to 
predict the future. The old management methods are not up to 
the task. Planning and forecasting are only accurate when based 
on a long, stable operating history and a relatively static envi- 
ronment. Startups have neither. 

The second problem is that after seeing traditional man- 
agement fail to solve this problem, some entrepreneurs and 
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investors have thrown up their hands and adopted the “Just Do © 
It" school of startups. This school believes that if management is 
the problem, chaos is the answer. Unfortunately, as I can attest 
firsthand, this doesn’t work either. 

It'may seem counterintuitive to think that something as dis- 
ruptive, innovative, and chaotic as a startup can be managed or, 
to be accurate, must be managed. Most people think of process 
and management as boring and dull, whereas startups are dy- 
namic and exciting. But what is actually exciting is to see start- 
ups succeed and change the world. The passion, energy, and 
vision that people bring to these new ventures are resources too 
precious to waste. We can—and must—do better. This book is 
about how. 


HOW THIS BOOK 1$ ORGANIZED 


This book is divided into three parts: “Vision,” “Steer,” and 
“Accelerate.” 

“Vision” makes the case for a new discipline of entrepre- 
neurial management. I identify who is an entrepreneur, de- 
fine a startup, and articulate a new way for startups to gauge if — 
they are making progress, called validated learning. To achieve 
that learning, we'll see that startups—in a garage or inside an 
enterprise—can use scientific experimentation to discover how 
to build a sustainable business. 

“Steer” dives into the Lean Startup method in detail, showing 
one major turn through the core Build-Measure-Learn feedback 
loop. Beginning with leap-of-faith assumptions that cry out for 
rigorous testing, youll learn how to build a minimum viable 
product to test those assumptions, a new accounting system for 
evaluating whether you're making progress, and a method for 
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deciding whether to pivot (changing course with one foot an- 
chored to the ground) or persevere. 

In “Accelerate,” we'll explore techniques that enable Lean 
Startups to speed through the Build-Measure-Learn feedback 
loop as quickly as possible, even as they scale. We'll explore lean 
manufacturing concepts that are applicable to startups, too, 
such as the power of small batches. We'll also discuss organi- 
zational design, how products grow, and how to apply Lean 
Startup principles beyond the proverbial garage, even inside the 
world’s largest companies. 


MANAGEMENT'S SECOND CENTURY 


As a society, we have a proven set of techniques for managing big 
companies and we know the best practices for building physical 
products. But when it comes to startups and innovation, we are 
still shooting in the dark. We are relying on vision, chasing the 
“great men” who can make magic happen, or trying to analyze 
our new products to death. These are new problems, born of the 
success of management in the twentieth century. 

This book attempts to put entrepreneurship and innovation 
on a rigorous footing. We are at the dawn of management's sec- 
ond century. It is our challenge to do something great with the 
opportunity we have been given. The Lean Startup movement 
seeks to ensure that those of us who long to build the next big 
thing will have the tools we need to change the world. 
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ENTREPRENEURIAL MANAGEMENT 


a a startup IS an exercise in institution building; thus, it 
necessarily involves management. This often comes as a sur- 
prise to aspiring entrepreneurs, because their associations with 
these two words are so diametrically opposed. Entrepreneurs are 
rightly wary of implementing traditional management practices 
early on in a startup, afraid that they will invite bureaucracy or 
stifle creativity. 

Entrepreneurs have been trying to fit the square peg of their 
unique problems into the round hole of general management 
for decades. As a result, many entrepreneurs take a “just do it” 
attitude, avoiding all forms of management, process, and disci- 
pline. Unfortunately, this approach leads to chaos more often 
than it does to success. I should know: my first startup failures 
were all of this kind. 

The tremendous success of general management over the last 
century has provided unprecedented material abundance, but 
those management principles are ill suited to handle the chaos 
and uncertainty that startups must face. 
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I believe that entrepreneurship requires a managerial discipline 
to harness the entrepreneurial opportunity we have been given. 

There are more entrepreneurs operating today than at any 
previous time in history. This has been made possible by dra- 
matic changes in the global economy. To cite but one example, 
one often hears commentators lament the loss of manufacturing 
jobs in the United States over the previous two decades, but one 
rarely hears about a corresponding loss of manufacturing capa- 
bility. That’s because total manufacturing output in the United 
States is increasing (by 15 percent in the last decade) even as jobs 
continue to be lost (see the charts below). In effect, the huge 
productivity increases made possible by modern management 
and technology have created more productive capacity than 
firms know what to do with.! 

We are living through an unprecedented worldwide entre- 
preneurial renaissance, but this opportunity is laced with peril. 
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Because we lack a coherent management paradigm for new in- 
novative ventures, were throwing our excess capacity around 
with wild abandon. Despite this lack of rigor, we are finding 
some ways to make money, but for every success there are far too 
many failures: products pulled from shelves mere weeks after 
being launched, high-profile startups lauded in the press and 
forgotten a few months later, and new products that wind up 
being used by nobody. What makes these failures particularly 
painful is not just the economic damage done to individual em- 
ployees, companies, and investors; they are also a colossal waste 
of our civilization’s most precious resource: the time, passion, 
and skill of its people. The Lean Startup movement is dedicated 
to preventing these failures. 


THE ROOTS OF THE LEAN STARTUP 


The Lean Startup takes its name from the lean manufacturing 
revolution that Taiichi Ohno and Shigeo Shingo are credited 
with developing at Toyota. Lean thinking is radically altering the 
way supply chains and production systems are run. Among its 
tenets are drawing on the knowledge and creativity of individual — 
workers, the shrinking of batch sizes, just-in-time production 
and inventory control, and an acceleration of cycle times. It 
taught the world the difference between value-creating activities 
and waste and showed how to build quality into products from 
the inside out. 

The Lean Startup adapts these ideas to the context of entre- 
preneurship, proposing that entrepreneurs judge their progress 
differently from the way other kinds of ventures do. Progress in 
manufacturing is measured by the production of high-quality 
physical goods. As we'll see in Chapter 3, the Lean Startup 
uses a different unit of progress, called validated learning. With 
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scientific learning as our yardstick, we can discover and elimi- 
nate the sources of waste that are plaguing entrepreneurship. 

A comprehensive theory of entrepreneurship should address 
all the functions of an early-stage venture: vision and concept, © 
product development, marketing and sales, scaling up, partner- 
ships and distribution, and structure and organizational de- 
sign: It has to provide a method for measuring progress in the 
context of extreme uncertainty. It can give entrepreneurs clear 
guidance on how to make the many trade-off decisions they 
face: whether and when to invest in process; formulating, plan- 
ning, and creating infrastructure; when to go it alone and when 
to partner; when to respond to feedback and when to stick 
with vision; and how and when to invest in scaling the busi- 
ness. Most of all, it must allow entrepreneurs to make testable 
predictions. 

For example, consider the recommendation that you build | 
cross-functional teams and hold them accountable to what we 
call learning milestones instead of organizing your company into 
strict functional departments (marketing, sales, information 
technology, human resources, etc.) that hold people accountable 
for performing well in their specialized areas (see Chapter 7). 
Perhaps you agree with this recommendation, or perhaps you 
are skeptical. Either way, if you decide to implement it, I predict 
that you pretty quickly will get feedback from your teams that 
the new process is reducing their productivity. They will ask to 
go back to the old way of working, in which they had the op- 
portunity to “stay efficient” by working in larger batches and 
passing work between departments. 

It’s safe to predict this result, and not just because I have seen 
it many times in the companies I work with. It is a straightfor- 
ward prediction of the Lean Startup theory itself. When people 
are used to evaluating their productivity locally, they feel that a 
good day is one in which they did their job well all day. When I 
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worked as a programmer, that meant eight straight hours of pro- 
gramming without interruption. That was a good day. In con- 
trast, if I was interrupted with questions, process, or—heaven 
~ forbid—meetings, I felt bad. What did I really accomplish that 
day? Code and product features were tangible to me; I could see 
them, understand them, and show them off. Learning, by con- 
trast, is frustratingly intangible. 

The Lean Startup asks people to start measuring their pro- 
ductivity differently. Because startups often accidentally build 
something nobody wants, it doesn’t matter much if they do it 
on time and on budget. The goal of a startup is to figure out 
the right thing to build—the thing customers want and will pay 
for—as quickly as possible. In other words, the Lean Startup is a 
new way of looking at the development of innovative new prod- 
ucts that emphasizes fast iteration and customer insight, a huge 
vision, and great ambition, all at the same time. 


000 


Henry Ford is one of the most successful and celebrated entrepre- 
neurs of all time. Since the idea of management has been bound 
up with the history of the automobile since its first days, I believe 
it is fitting to use the automobile as a metaphor for a startup. 

An internal combustion automobile is powered by two im- 
portant and very different feedback loops. The first feedback loop 
is deep inside the engine. Before Henry Ford was a famous CEO, 
he was an engineer. He spent his days and nights tinkering in his 
garage with the precise mechanics of getting the engine cylinders 
to move. Each tiny explosion within the cylinder provides the 
motive force to turn the wheels but also drives the ignition of the 
next explosion. Unless the timing of this feedback loop is man- 
aged precisely, the engine will sputter and break down. 

Startups have a similar engine that I call the engine of growth. 
The markets and customers for startups are diverse: a toy 
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company, a consulting firm, and a manufacturing plant may not 
seem like they have much in common, but, as we'll see, they 
operate with the same engine of growth. 

Every new version of a product, every new feature, and every 
new marketing program is an attempt to improve this engine 
of growth. Like Henry Ford’s tinkering in his garage, not all 
of these changes turn out to be improvements. New product 
development happens in fits and starts. Much of the time in 
a startup’s life is spent tuning the engine by making improve- 
ments in product, marketing, or operations. 

The second important feedback loop in an automobile is be- 
tween the driver and the steering wheel. This feedback is so im- 
mediate and automatic that we often don’t think about it, but 
it is steering that differentiates driving from most other forms 
of transportation. If you have a daily commute, you probably 
know the route so well that your hands seem to steer you there 
on their own accord. We can practically drive the route in our 
sleep. Yet if I asked you to close your eyes and write down exactly 
how to get to your office—not the street directions but every ac- 
tion you need to take, every push of hand on wheel and foot on 
pedals—youd find it impossible. The choreography of driving is 
incredibly complex when one slows down to think about it. 

By contrast, a rocket ship requires just this kind of in-advance 
calibration. It must be launched with the most precise instruc- 
tions on what to do: every thrust, every firing of a booster, 
and every change in direction. The tiniest error at the point of 
launch could yield catastrophic results thousands of miles later. 

Unfortunately, too many startup business plans look more 
like they are planning to launch a rocket ship than drive a car. 
They prescribe the steps to take and the results to expect in ex- 
cruciating detail, and as in planning to launch a rocket, they are 
set up in such a way that even tiny errors in assumptions can 
lead to catastrophic outcomes. 
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One company I worked with had the misfortune of forecast- 
ing significant customer adoption—in the millions—for one of 
its new products. Powered by a splashy launch, the company 
successfully executed its plan. Unfortunately, customers did not 
flock to the product in great numbers. Even worse, the company 
had invested in massive infrastructure, hiring, and support to 
handle the influx of customers it expected. When the customers 
failed to materialize, the company had committed itself so com- 
pletely that they could not adapt in time. They had “achieved 
failure’—successfully, faithfully, and rigorously executing a plan 
that turned out to have been utterly flawed. 

The Lean Startup method, in contrast, is designed to teach 
you how to drive a startup. Instead of making complex plans that 
are based on a lot of assumptions, you can make constant ad- 
justments with a steering wheel called the Build-Measure-Learn 
feedback loop. Through this process of steering, we can learn 
when and if it’s time to make a sharp turn called a pivot or 
whether we should persevere along our current path. Once we 
have an engine that’s revved up, the Lean Startup offers methods 
to scale and grow the business with maximum acceleration. 

Throughout the process of driving, you always have a clear 
idea of where youre going. If you're commuting to work, you 
don’t give up because there’s a detour in the road or you made a 
wrong turn. You remain thoroughly focused on getting to your 
destination. | 

Startups also have a true north, a destination in mind: creat- 
ing a thriving and world-changing business. I call that a startup’s 
vision. To achieve that vision, startups employ a strategy, which 
includes a business model, a product road map, a point of view 
about partners and competitors, and ideas about who the cus- 
tomer will be. The product is the end result of this strategy (see 
the chart on page 23). 
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PRODUCT 


Products change constantly through the process of optimiza- 
tion, what I call tuning the engine. Less frequently, the strategy 
may have to change (called a pivot). However, the overarching 
vision rarely changes. Entrepreneurs are committed to seeing 
the startup through to that destination. Every setback is an op- 
portunity for learning how to get where they want to go (see the 
chart below). 


CHANGE 


PRODUCT Optimization 


Pivot 
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In real life, a startup is a portfolio of activities. A lot is hap- 
pening simultaneously: the engine is running, acquiring new 
customers and serving existing ones; we are tuning, trying to 
improve our product, marketing, and operations; and we are 
steering, deciding if and when to pivot. The challenge of en- 
trepreneurship is to balance all these activities. Even the small- 
est startup faces the challenge of supporting existing customers 
while trying to innovate. Even the most established company 
faces the imperative to invest in innovation lest it become obso- 
lete. As companies grow, what changes is the mix of these activi- 
ties in the company’s portfolio of work. 
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Entrepreneurship is management. And yet, imagine a modern 
manager who is tasked with building a new product in the con- 
text of an established company. Imagine that she goes back to 
her company’s chief financial officer (CFO) a year later and says, 
“We have failed to meet the growth targets we predicted. In fact, 
we have almost no new customers and no new revenue. How- 
ever, we have learned an incredible amount and are on the cusp 
of a breakthrough new line of business. All we need is another 
year.” Most of the time, this would be the last report this intra- 
preneur would give her employer. The reason is that in general 
management, a failure to deliver results is due to either a fail- 
ure to plan adequately or a failure to execute properly. Both are 
significant lapses, yet new product development in our modern 
economy routinely requires exactly this kind of failure on the 
way to greatness. In the Lean Startup movement, we have come 
to realize that these internal innovators are actually entrepre- 
neurs, too, and that entrepreneurial management can help them 
succeed; this is the subject of the next chapter. 
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WHO, EXACTLY, 1S AN ENTREPRENEUR? 


\' | travel the world talking about the Lean Startup, I’m consis- 
tently surprised that I meet people in the audience who seem 
out of place. In addition to the more traditional startup en- 
trepreneurs I meet, these people are general managers, mostly 
_ working in very large companies, who are tasked with creating 
new ventures or product innovations. They are adept at organi- 
zational politics: they know how to form autonomous divisions 
with separate profit and loss statements (P8&Ls) and can shield 
controversial teams from corporate meddling. The biggest sur- 
prise is that they are visionaries. Like the startup founders I have 
worked with for years, they can see the future of their industries 
and are prepared to take bold risks to seek out new and innova- 
tive solutions to the problems their companies face. 

Mark, for example, is a manager for an extremely large com- 
pany who came to one of my lectures. He is the leader of a 
division that recently had been chartered to bring his company 
into the twenty-first century by building a new suite of products 
designed to take advantage of the Internet. When he came to 
talk to me afterward, I started to give him the standard advice 
about how to create innovation teams inside big companies, and 
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he stopped me in midstream: “Yeah, I’ve read The Innovator’ 
Dilemma.' ve got that all taken care of.” He was a long-term 
employee of the company and a successful manager to boot, 
sO managing internal politics was the least of his problems. I 
should have known; his success was a testament to his ability to 
navigate the company’s corporate policies, personnel, and pro- 
cesses to get things done. 

Next, I tried to give him some advice about the future, about 
cool new highly leveraged product development technologies. 
He interrupted me again: “Right. I know all about the Internet, 
and I have a vision for how our company needs to adapt to it 
or die.” 

Mark has all the entrepreneurial eis nailed—proper 
team structure, good personnel, a strong vision for the future, 
and an appetite for risk taking—and so it finally occurred to me 
to ask why he was coming to me for advice. He said, “It’s as if we 
have all of the raw materials: kindling, wood, paper, flint, even 
some sparks. But where's the fire?” The theories of management 
that Mark had studied treat innovation like a “black box” by fo- 
cusing on the structures companies need to put in place to form 
internal startup teams. But Mark found himself working inside 
the black box—and in need of guidance. 

What Mark was missing was a process for converting the raw 
materials of innovation into real-world breakthrough successes. 
Once a team is set up, what should it do? What process should 
it use? How should it be held accountable to performance mile- 
stones? These are questions the Lean Startup methodology i is de- 
signed to answer. 

My point? Mark is an entrepreneur just like a Silicon Valley 
high-tech founder with a garage startup. He needs the principles 
of the Lean Startup just as much as the folks I thought of as clas- 
sic entrepreneurs do. 

Entrepreneurs who operate inside an established organization 
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sometimes are called “intrapreneurs” because of the special cir- 
cumstances that attend building a startup within a larger com- 
pany. As I have applied Lean Startup ideas in an ever-widening 
variety of companies and industries, I have come to believe that 
intrapreneurs have much more in common with the rest of the 
community of entrepreneurs than most people believe. Thus, 
when I use the term entrepreneur, | am referring to the whole 
startup ecosystem regardless of company size, sector, or stage of 
development. 

This book is for entrepreneurs of all stripes: from young 
visionaries with little backing but great ideas to seasoned vision- 
aries within larger companies such as Mark—and the people 
who hold them accountable. 


IF 1M AN ENTREPRENEUR, WHAT'S A STARTUP? 


The Lean Startup is a set of practices for helping entrepreneurs 
increase their odds of building a successful startup. To set the 
record straight, it’s important to define what a startup Is: 


A startup is a human institution designed to create a 
new product or service under conditions of extreme 
uncertainty. 


I’ve come to realize that the most important part of this defi- 
nition is what it omits. It says nothing about size of the com- 
pany, the industry, or the sector of the economy. Anyone who is 
creating a new product or business under conditions of extreme 
uncertainty is an entrepreneur whether he or she knows it or not 
and whether working in a government agency, a venture-backed 
company, a nonprofit, or a decidedly for-profit company with 
financial investors. 
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Let's take a look at each of the pieces. The word institution 
connotes bureaucracy, process, even lethargy. How can that be 
part of a startup? Yet successful startups are full of activities as- 
sociated with building an institution: hiring creative employees, 
coordinating their activities, and creating a company culture 
that delivers results. 

We often lose sight of the fact that a startup is not just about 
a product, a technological breakthrough, or even a brilliant idea. 
A startup is greater than the sum of its parts; it is an acutely 
human enterprise. 

The fact that a startup’s product or service is a new innovation 
is also an essential part of the definition and a tricky part too. I 
prefer to use the broadest definition of product, one that encom- 
passes any source of value for the people who become custom- 
ers. Anything those customers experience from their interaction 
with a company should be considered part of that company’s 
product. This is true of a grocery store, an e-commerce website, 
a consulting service, and a nonprofit social service agency. In 
every case, the organization is dedicated to uncovering a new 
source of value for customers and cares about the impact of its 
product on those customers. 

It's also important that the word innovation be understood 
broadly. Startups use many kinds of innovation: novel scientific 
discoveries, repurposing an existing technology for a new use, 
devising a new business model that unlocks value that was hid- 
den, or simply bringing a product or service to a new location 
or a previously underserved set of customers. In all these cases, 
innovation is at the heart of the company’s success. 

There is one more important part of this definition: the con- 
text in which the innovation happens. Most businesses—large 
and small alike—are excluded from this context. Startups are 
designed to confront situations of extreme uncertainty. To open 
up a new business that is an exact clone of an existing business all 
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the way down to the business model, pricing, target customer, 
and product may be an attractive economic investment, but it is 
not a startup because its success depends only on execution—so 
much so that this success can be modeled with high accuracy. 
(This is why so many small businesses can be financed with sim- 
ple bank loans; the level of risk and uncertainty is understood 
well enough that a loan officer can assess its prospects.) 

Most tools from general management are not designed to 
flourish in the harsh soil of extreme uncertainty in which start- 
ups thrive. The future is unpredictable, customers face a growing 
array of alternatives, and the pace of change is ever increasing. 
Yet most startups—in garages and enterprises alike—still are 
managed by using standard forecasts, product milestones, and 
detailed business plans. 


THE SNAPTAX STORY 


In 2009, a startup decided to try something really audacious. 
They wanted to liberate taxpayers from expensive tax stores by 
automating the process of collecting information typically found 
on W-2 forms (the end-of-year statement that most employ- 
ees receive from their employer that summarizes their taxable 
wages for the year). The startup quickly ran into difficulties. 
Even though many consumers had access to a printer/scanner in 
their home or office, few knew how to use those devices. After 
numerous conversations with potential customers, the team 
lit upon the idea of having customers take photographs of the 
forms directly from their cell phone. In the process of testing this 
concept, customers asked something unexpected: would it be 
possible to finish the whole tax return right on the phone itself? 
That was not an easy task. Traditional tax preparation re- 
quires consumers to wade through hundreds of questions, many 
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forms, and a lot of paperwork. This startup tried something 
novel by deciding to ship an early version of its product that 
could do much less than a complete tax package. The initial 
version worked only for consumers with a very simple return to 
file, and it worked only in California. 

Instead of having consumers fill out a complex form, they al- 
lowed the customers to use the phone’s camera to take a picture 
of their W-2 forms. From that single picture, the company devel- 
oped the technology to compile and file most of the 1040 EZ tax 
return. Compared with the drudgery of traditional tax filing, the 
new product—called SnapTax—provides a magical experience. 
From its modest beginning, SnapTax grew into a significant 
startup success story. Its nationwide launch in 2011 showed that 
customers loved it, to the tune of more than 390, 000 downloads 
in the first three weeks. 

This is the kind of amazing innovation youd expect from a 
new startup. 

However, the name of this company may surprise you. 
Snap Tax was developed by Intuit, America’s largest producer 
of finance, tax, and accounting tools for individuals and small 
businesses. With more than 7,700 employees and annual rev- 
enues in the billions, Intuit is not a typical startup.* 

The team that built SnapTax doesn’t look much like the ar- 
chetypal image of entrepreneurs either. They don't work in a 
garage or eat ramen noodles. Their company doesn’t lack for re- 
sources. They are paid a full salary and benefits. They come into 
a regular office every day. Yet they are entrepreneurs. 

Stories like this one are not nearly as common inside large 
corporations as they should be. After all, SnapTax competes di- 
rectly with one of Intuit’s flagship products: the fully featured 
TurboTax desktop software. Usually, companies like Intuit 
fall into the trap described in Clayton Christensten’s The In- 


novators Dilemma: they are very good at creating incremental 
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improvements to existing products and serving existing custom- 
ers, which Christensen called sustaining innovation, but struggle 
to create breakthrough new products—disruptive innovation— 
that can create new sustainable sources of growth. 

One remarkable part of the SnapTax story is what the team 
leaders said when I asked them to account for their unlikely 
success. Did they hire superstar entrepreneurs from outside the 
company? No, they assembled a team from within Intuit. Did 
they face constant meddling from senior management, which 
is the bane of innovation teams in many companies? No, their 
executive sponsors created an “island of freedom” where they 
could experiment as necessary. Did they have a huge team, a 
large budget, and lots of marketing dollars? Nope, they started 
with a team of five. 

What allowed the SnapTax team to innovate was not their 
genes, destiny, or astrological signs but a process deliberately facili- 
tated by Intuit’s senior management. Innovation is a bottoms-up, 
decentralized, and unpredictable thing, but that doesn’t mean it 
cannot be managed. It can, but to do so requires a new man- 
agement discipline, one that needs to be mastered not just by 
practicing entrepreneurs seeking to build the next big. thing but 
also by the people who support them, nurture them, and hold 
them accountable. In other words, cultivating entrepreneurship 
is the responsibility of senior management. Today, a cutting-edge 
company such as Intuit can point to success stories like SnapTax 
because it has recognized the need for a new management para- 
digm. This is a realization that was years in the making.° 


A SEVEN-THOUSAND-PERSON LEAN STARTUP 


In 1983, Intuit’s founder, the legendary entrepreneur Scott 
Cook, had the radical notion (with cofounder Tom Proulx) 


92 THE LEAN STARTUP 


that personal accounting should happen by computer. Their 
success was far from inevitable; they faced numerous competi- 
tors, an uncertain future, and an initially tiny market. A de- 
cade later, the company went public and subsequently fended 
off well-publicized attacks from larger incumbents, includ- 
ing the software behemoth Microsoft. Partly with the help of 
famed venture capitalist John Doerr, Intuit became a fully di- 
versified enterprise, a member of the Fortune 1000 that now 
provides dozens of market-leading products across its major 
divisions. 

This is the kind of entrepreneurial success we're used to hear- 
ing about: a ragtag team of underdogs who eventually achieve 
fame, acclaim, and significant riches. 

Flash-forward to 2002. Cook was frustrated. He had just 
tabulated ten years of data on all of Intuit’s new product intro- 
ductions and had concluded that the company was getting a 
measly return on its massive investments. Simply put, too many 
of its new products were failing. By traditional standards, Intuit 
is an extremely well-managed company, but as Scott dug into 
the root causes of those failures, he came to a difficult conclu- 
sion: the prevailing management paradigm he and his company 
had been practicing was inadequate to the problem of continu- 
ous innovation in the modern economy. 

By fall 2009, Cook had been working to change Intuit’s 
management culture for several years. He came across my early 
work on the Lean Startup and asked me to give a talk at In- 
tuit. In Silicon Valley this is not the kind of invitation you turn 
down. I admit I was curious. I was still at the beginning of my 
Lean Startup journey and didn't have much appreciation for the 
challenges faced by a Fortune 1000 company like his. 

My conversations with Cook and Intuit chief executive of- 
ficer (CEO) Brad Smith were my initiation into the thinking of 
modern general managers, who struggle with entrepreneurship 
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every bit as much as do venture capitalists and founders in a 
garage. To combat these challenges, Scott and Brad are going 
back to Intuit’s roots. They are working to build entrepreneur- 
ship and risk taking into all their divisions. 

For example, consider one of Intuit’s flagship products. Be- 
cause TurboTax does most of its sales around tax season in the 
United States, it used to have an extremely conservative culture. 
Over the course of the year, the marketing and product teams 
would conceive one major initiative that would be rolled out 
just in time for tax season. Now they test over five hundred dif- 
ferent changes in a two-and-a-half-month tax season. They're 
running up to seventy different tests per week. The team can 
make a change live on its website on Thursday, run it over the 
weekend, read the results on Monday, and come to conclusions 
starting Tuesday; then they rebuild new tests on Thursday and 
launch the next set on Thursday night. 

As Scott put it, “Boy, the amount of learning they get is 
just immense now. And what it does is develop entrepreneurs, 
because when you have only one test, you dont have entrepre- 
neurs, you have politicians, because you have to sell. Out of a 
hundred good ideas, you've got to sell your idea. So you build 
up a society of politicians and salespeople. When you have five 
hundred tests you're running, then everybody’s ideas can run. 
And then you create entrepreneurs who run and learn and can 
retest and relearn as opposed to a society of politicians. So we're 
trying to drive that throughout our organization, using exam- 
ples which have nothing to do with high tech, like the website 
example. Every business today has a website. You don't have to 
be high tech to use fast-cycle testing.” 

This kind of change is hard. After all, the company has a sig- 
nificant number of existing customers who continue to demand 
exceptional service and investors who expect steady, growing 
returns. 
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Scott says, 


It goes against the grain of what people have been taught 
in business and what leaders have been taught. The prob- 
lem isn’t with the teams or the entrepreneurs. They love 
the chance to quickly get their baby out into the market. 
They love the chance to have the customer vote instead 
of the suits voting. The real issue is with the leaders and 
the middle managers. There are many business leaders 
who have been successful because of analysis. They think 
they’re analysts, and their job is to do great planning and 
analyzing and have a plan. 


The amount of time a company can count on holding on 
to market leadership to exploit its earlier innovations is shrink- 
ing, and this creates an imperative for even the most entrenched 
companies to invest in innovation. In fact, I believe a compa- 
ny's only sustainable path to long-term economic growth is to 
build an “innovation factory” that uses Lean Startup techniques 
to create disruptive innovations on a continuous basis. In other 
words, established companies need to figure out how to accom- 
plish what Scott Cook did in 1983, but on an industrial scale 
and with an established cohort of managers steeped in tradi- 
tional management culture. 

Ever the maverick, Cook asked me to put these ideas to the test, 
and so I gave a talk that was simulcast to all seven thousand—plus 
Intuit employees during which I explained the theory of the 
Lean Startup, repeating my definition: an organization designed 
to create new products and services under conditions of extreme 
uncertainty. 

What happened next is etched in my memory. CEO Brad 
Smith had been sitting next to me as I spoke. When I was done, 
he got up and said before all of Intuit’s employees, “Folks, listen 
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up. You heard Eric’s definition of a startup. It has three parts, 
and we here at Intuit match all three parts of that definition.” 

Scott and Brad are leaders who realize that something new is 
needed in management thinking. Intuit is proof that this kind 
of thinking can work in established companies. Brad explained 
to me how they hold themselves accountable for their new in- 
novation efforts by measuring two things: the number of cus- 
tomers using products that didn’t exist three years ago and the 
percentage of revenue coming from offerings that did not exist 
three years ago. 

Under the old model, it took an average of 5.5 years for a 
successful new product to start generating $50 million in rev- 
enue. Brad explained to me, “We've generated $50 million in 
offerings that did not exist twelve months ago in the last year. 
Now it’s not one particular offering. It’s a combination of a 
whole bunch of innovation happening, but that’s the kind of 
stuff that’s creating some energy for us, that we think we can 
truly short-circuit the ramp by killing things that don’t make 
sense fast and doubling down on the ones that do.” For a com- 
pany as large as Intuit, these are modest results and early days. 
They have decades of legacy systems and legacy thinking to 
overcome. However, their leadership in adopting entrepreneur- 
ial management is starting to pay off. 

Leadership requires creating conditions that enable employ- 
ees to do the kinds of experimentation that entrepreneurship re- 
quires. For example, changes in TurboTax enabled the Intuit team 
to develop five hundred experiments per tax season. Before thar, 
marketers with great ideas couldnt have done those tests even 
if they'd wanted to, because they didn’t have a system in place 
through which to change the website rapidly. Intuit invested in 
systems that increased the speed at which tests could be built, de- 
ployed, and analyzed. 

As Cook says, “Developing these experimentation systems 1s 
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the responsibility of senior management; they have to be put in 
by the leadership. It’s moving leaders from playing Caesar with 
their thumbs up and down on every idea to—instead—putting 
in the culture and the systems so that teams can move and in- 
novate at the speed of the experimentation system.” 





LEARN 


\' an entrepreneur, nothing plagued me more than the question of 
whether my company was making progress toward creating 
a successful business. As an engineer and later as a manager, I 
was accustomed to measuring progress by making sure our work 
proceeded according to plan, was high quality, and cost about 
what we had projected. 

After many years as an entrepreneur, I started to worry about 
measuring progress in this way. What if we found ourselves 
building something that nobody wanted? In that case what did 
it matter if we did it on time and on budget? When I went home 
at the end of a day’s work, the only things I knew for sure were 
that I had kept people busy and spent money that day. I hoped 
that my team’s efforts took us closer to our goal. If we wound up 
taking a wrong turn, I’d have to take comfort in the fact that at 
least we'd learned something important. 

Unfortunately, “learning” is the oldest excuse in the book for 
a failure of execution. It’s what managers fall back on when they 
fail to achieve the results we promised. Entrepreneurs, under 
pressure to succeed, are wildly creative when it comes to demon- 
strating what we have learned. We can all tell a good story when 
our job, career, or reputation depends on it. 

However, learning is cold comfort to employees who are fol- 
lowing an entrepreneur into the unknown. It is cold comfort to 
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the investors who allocate precious money, time, and energy to en- 
trepreneurial teams. It is cold comfort to the organizations—large 
and small—that depend on entrepreneurial innovation to survive. 
You can’t take learning to the bank; you can’t spend it or invest it. 
You cannot give it to customers and cannot return it to limited 
partners. Is it any wonder that learning has a bad name in entre- 
preneurial and managerial circles? 

Yet if the fundamental goal of entrepreneurship is to engage 
in organization building under conditions of extreme uncer- 
tainty, its most vital function is learning. We must learn the 
truth about which elements of our strategy are working to re- 
alize our vision and which are just crazy. We must learn what 
customers really want, not what they say they want or what we 
think they should want. We must discover whether we are on a 
path that will lead to growing a sustainable business. 

In the Lean Startup model, we are rehabilitating learning 
with a concept I call validated learning. Validated learning is not 
after-the-fact rationalization or a good story designed to hide 
failure. It is a rigorous method for demonstrating progress when 
one is embedded in the soil of extreme uncertainty in which 
startups grow. Validated learning is the process of demonstrating 
empirically that a team has discovered valuable truths about a 
startups present and future business prospects. It is more con- 
crete, more accurate, and faster than market forecasting or clas- 
sical business planning. It is the principal antidote to the lethal 
problem of achieving failure: successfully executing a plan that 
leads nowhere. 


VALIDATED LEARNING AT IMVU 


Let me illustrate this with an example from my career. Many 
audiences have heard me recount the story of IMVU’s founding 


Learn 39 


and the many mistakes we made in developing our first product. 
l’ll now elaborate on one of those mistakes to illustrate validated 
learning clearly. 

Those of us involved in the founding of IMVU aspired to 
be serious strategic thinkers. Each of us had participated in 
previous ventures that had failed, and we were loath to repeat 
that experience. Our main concerns in the early days dealt with 
the following questions: What should we build and for whom? 
What market could we enter and dominate? How could we 
build durable value that would not be subject to erosion by 
competition?’ 


Brilliant Strategy 


We decided to enter the instant messaging (IM) market. In 
2004, that market had hundreds of millions of consumers ac- 
tively participating worldwide. However, the majority of the 
customers who were using IM products were not paying for 
the privilege. Instead, large media and portal companies such 
as AOL, Microsoft, and Yahoo! operated their IM networks as 
a loss leader for other services while making modest amounts of 
money through advertising. 

IM is an example of a market that involves strong network 
effects. Like most communication networks, IM is thought to 
follow Metcalfe’s law: the value of a network as a whole is pro- 
portional to the square of the number of participants. In other 
words, the more people in the network, the more valuable the 
network. This makes intuitive sense: the value to each partici- 
pant is driven primarily by how many other people he or she can 
communicate with. Imagine a world in which you own the only 
telephone; it would have no value. Only when other people also 
have a telephone does it become valuable. 

In 2004, the IM market was locked up by a handful of 
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incumbents. The top three networks controlled more than 80 
percent of the overall usage and were in the process of consoli- 
dating their gains in market share at the expense of a number of 
smaller players.” The common wisdom was that it was more or 
less impossible to bring a new IM network to market without 
spending an extraordinary amount of money on marketing. 

The reason for that wisdom is simple. Because of the power 
of network effects, IM products have high switching costs. To 
switch from one network to another, customers would have to 
convince their friends and colleagues to switch with them. This 
extra work for customers creates a barrier to entry in the IM 
market: with all consumers locked in to an incumbent’s product, 
there are no customers left with whom to establish a beachhead. 

At IMVU we settled on a strategy of building a product that 
would combine the large mass appeal of traditional IM with 
the high revenue per customer of three-dimensional (3D) video 
games and virtual worlds. Because of the near impossibility of 
bringing a new IM network to market, we decided to build an 
IM add-on product that would interoperate with the existing 
networks. Thus, customers would be able to adopt the IMVU 
virtual goods and avatar communication technology with- 
out having to switch IM providers, learn a new user interface, 
and—most important—bring their friends with them. 

In fact, we thought this last point was essential. For the 
add-on product to be useful, customers would have to use it 
_ with their existing friends. Every communication would come 
embedded with an invitation to join IMVU. Our product 
would be inherently viral, spreading throughout the existing 
IM networks like an epidemic. To achieve that viral growth, 
it was important that our add-on product support as many of 
the existing IM networks as possible and work on all kinds of 
computers. 
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Six Months to Launch 


With this strategy in place, my cofounders and I began a pe- 
riod of intense work. As the chief technology officer, it was my 
responsibility, among other things, to write the software that 
would support IM interoperability across networks. My co- 
founders and I worked for months, putting in crazy hours strug- 
gling to get our first product released. We gave ourselves a hard 
deadline of six months—180 days—to launch the product and 
attract our first paying customers. It was a grueling schedule, 
but we were determined to launch on time. 

The add-on product was so large and complex and had so 
many moving parts that we had to cut a lot of corners to get it 
done on time. I won't mince words: the first version was terrible. 
We spent endless hours arguing about which bugs to fix and 
which we could live with, which features to cut and which to try 
to cram in. It was a wonderful and terrifying time: we were full 
of hope about the possibilities for success and full of fear about 
the consequences of shipping a bad product. 

Personally, I was worried that the low quality of the product 
would tarnish my reputation as an engineer. People would think 
I didn’t know how to build a quality product. All of us feared 
tarnishing the IMVU brand; after all, we were charging people 
money for a product that didn’t work very well. We all envi- 
sioned the damning newspaper headlines: “Inept Entrepreneurs 
Build Dreadful Product.” 

As launch day approached, our fears escalated. In our situa- 
tion, many entrepreneurial teams give in to fear and postpone 
the launch date. Although I understand this impulse, I am glad 
we persevered, since delay prevents many startups from getting 
the feedback they need. Our previous failures made us more 
afraid of another, even worse, outcome than shipping a bad 
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product: building something that nobody wants. And so, teeth 
clenched and apologies at the ready, we released our product to 


the public. 


‘Launch 


And then—nothing happened! It turned out that our fears were 
unfounded, because nobody even tried our product. At first I 
was relieved because at least nobody was finding out how bad 
the product was, but soon that gave way to serious frustration. 
After all the hours we had spent arguing about which features 
to include and which bugs to fix, our value proposition was so 
far off that customers weren't getting far enough into the experi- 
ence to find out how bad our design choices were. Customers 
wouldn't even download our product. 

Over the ensuing weeks and months, we labored to make 
the product better. We brought in a steady flow of custom- 
ers through our online registration and download process. We 
treated each day’s customers as a brand-new report card to let us 
know how we were doing. We eventually learned how to change 
the product's positioning so that customers at least would 
download it. We were making improvements to the underlying 
product continuously, shipping bug fixes and new changes daily. 
However, despite our best efforts, we were able to persuade only 
a pathetically small number of people to buy the product. 

In retrospect, one good decision we made was to set clear 
revenue targets for those early days. In the first month we in- 
tended to make $300 in total revenue, and we did—barely. 
Many friends and family members were asked (okay, begged). 
Each month our small revenue targets increased, first to $350 
and then to $400. As they rose, our struggles increased. We soon 
ran out of friends and family; our frustration escalated. We were 
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making the product better every day, yet our customers’ behav- 
ior remained unchanged: they still wouldn't use it. 

Our failure to move the numbers prodded us to accelerate our 
efforts to bring customers into our office for in-person interviews 
and usability tests. The quantitative targets created the motiva- 
tion: to engage in qualitative inquiry and guided us in the ques- 
tions we asked; this is a pattern we'll see throughout this book. 

I wish I could say that I was the one to realize our mistake 
and suggest the solution, but in truth, I was the last to admit 
the problem. In short, our entire strategic analysis of the mar- 
ket was utterly wrong. We figured this out empirically, through 
experimentation, rather than through focus groups or market 
research. Customers could not tell us what they wanted; most, 
after all, had never heard of 3D avatars. Instead, they revealed 
the truth through their action or inaction as we struggled to 
make the product better. 


Talking to Customers 


Out of desperation, we decided to talk to some potential cus- 
tomers. We brought them into our office, and said, “Try this 
new product; it’s IMVU.” If the person was a teenager, a heavy 
user of IM, or a tech early adopter, he or she would engage with 
us. In constrast, if it was a mainstream person, the response 
was, “Right. So exactly what would you like me to do?” We'd get 
nowhere with the mainstream group; they thought IMVU was 
too weird. 

Imagine a seventeen-year-old girl sitting down with us to 
look at this product. She chooses her avatar and says, “Oh, this 
is really fun.” She’s customizing the avatar, deciding how she's 
going to look. Then we say, “All right, it’s time to download the 
instant messaging add-on,” and she responds, “What's that?” 
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“Well, it's this thing that interoperates with the instant mes- 
saging client.” She's looking at us and thinking, “I’ve never heard 
of that, my friends have never heard of that, why do you want me 
to do that?” It required a lot of explanation; an instant messaging 
add-on was not a product category that existed in her mind. 

But since she was in the room with us, we were able to talk 
her into doing it. She downloads the product, and then we say, 
“Okay, invite one of your friends to chat.” And she says, “No 
way!” We say, “Why not?” And she says, “Well, I don’t know if 
this thing is cool yet. You want me to risk inviting one of my 
friends? What are they going to think of me? If it sucks, they’re 
going to think I suck, right?” And we say, “No, no, it’s going 
to be so much fun once you get the person in there; it’s a social 
product.” She looks at us, her face filled with doubt; you can 
see that this is a deal breaker. Of course, the first time I had that 
experience, I said, “It’s all right, it’s just this one person, send her 
away and get me a new one.” Then the second customer comes 
in and says the same thing. Then the third customer comes in, 
and it’s the same thing. You start to see patterns, and no matter 
how stubborn you are, there’s obviously something wrong. 

Customers kept saying, “I want to use it by myself. I want 
to try it out first to see if it’s really cool before I invite a friend.” 
Our team was from the video game industry, so we understood 
what that meant: single-player mode. So we built a single-player 
version. Wed bring new customers into our office. They'd cus- 
tomize the avatar and download the product like before. Then 
they would go into single-player mode, and we'd say, “Play with 
your avatar and dress it up; check out the cool moves it can 
make.” Followed by, “Okay, you did that by yourself; now it’s 
time to invite one of your friends.” You can see what’s com- 
ing. They'd say, “No way! This isn’t cool.” And we'd say, “Well, 
we told you it wasn't going to be cool! What is the point of a 
single-player experience for a social product?” See, we thought 
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we should get a gold star just for listening to our customers. Ex- 
cept our customers still didn’t like the product. They would look 
at us and say, “Listen, old man, you don't understand. What 
is the deal with this crazy business of inviting friends before I 
know if it’s cool?” It was a total deal breaker. 

Out of further desperation, we introduced a feature called 
ChatNow that allows you to push a button and be randomly 
matched with somebody else anywhere in the world. The only 
thing you have in common is that you both pushed the button 
at the same time. All of a sudden, in our customer service tests, 
people were saying, “Oh, this is fun!” 

So we'd bring them in, they'd use ChatNow, and maybe they 
would meet somebody they thought was cool. They'd say, “Hey, 
that guy was neat; I want to add him to my buddy list. Where's 
my buddy list?” And we'd say, “Oh, no, you don’t want a new 
buddy list; you want to use your regular AOL buddy list.” Re- 
member, this was how we planned to harness the interoperabil- 
ity that would lead to network effects and viral growth. Picture 
the customer looking at us, asking, “What do you want me to 
do exactly?” And we'd say, “Well, just give the stranger your 
AIM screen name so you can put him on your buddy list.” You 
could see their eyes go wide, and they'd say, “Are you kidding 
me? A stranger on my AIM buddy list?” To which we'd respond, 
“Yes; otherwise you'd have to download a whole new IM client 
with a new buddy list.” And they'd say, “Do you have any idea 
how many IM clients I already run?” 

“No. One or two, maybe?” That’s how many clients each of 
us in the office used. To which the teenager would say, “Duh! I 
run eight.” We had no idea how many instant messaging clients 
there were in the world. 

We had the incorrect preconception that it’s a challenge to 
learn new software and it’s tricky to move your friends over to a 
new buddy list. Our customers revealed that this was nonsense. 
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We wanted to draw diagrams on the whiteboard that showed 
why our strategy was brilliant, but our customers didn’t under- 
stand concepts like network effects and switching costs. If we 
tried to explain why they should behave the way we predicted, 
they'd just shake their heads at us, bewildered. 

We had a mental model for how people used software that 
was years out of date, and so eventually, painfully, after doz- 
ens of meetings like that, it started to dawn on us that the IM 
add-on concept was fundamentally flawed.’ 

Our customers did not want an IM add-on; they wanted a 
stand-alone IM network. They did not consider having to learn 
how to use a new IM program a barrier; on the contrary, our 
early adopters used many different IM programs simultane- 
ously. Our customers were not intimidated by the idea of having 
to take their friends with them to a new IM network; it turned 
out that they enjoyed that challenge. Even more surprising, 
our assumption that customers would want to use avatar-based 
IM primarily with their existing friends was also wrong. They 
wanted to make new friends, an activity that 3D avatars are par- 
ticularly well suited to facilitating. Bit by bit, customers tore 
apart our seemingly brilliant initial strategy. 


Throwing My Work Away 


Perhaps you can sympathize with our situation and forgive my 
obstinacy. After all, it was my work over the prior months that 
needed to be thrown away. I had slaved over the software that 
was required to make our IM program interoperate with other 
_ networks, which was at the heart of our original strategy. When 
it came time to pivot and abandon that original strategy, almost 
all of my work—thousands of lines of code—was thrown out. 
I felt betrayed. I was a devotee of the latest in software develop- 
ment methods (known collectively as agile development), which 
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promised to help drive waste out of product development. 
However, despite that, I had committed the biggest waste of all: 
building a product that our customers refused to use. That was 
really depressing. 

I wondered: in light of the fact that my work turned out to 
be a waste of time and energy, would the company have been 
just as well off if I had spent the last six months on a beach sip- 
ping umbrella drinks? Had I really been needed? Would it have 
been better if I had not done any work at all? 

There is, as I mentioned at the beginning of this chap- 
ter, always one last refuge for people aching to justify their 
own failure. I consoled myself that if we hadn't built this first 
product—mistakes and all—we never would have learned these 
important insights about customers. We never would have 
learned that our strategy was flawed. There is truth in this excuse: 
what we learned during those critical early months set IMVU 
on a path that would lead to our eventual breakout success. 

For a time, this “learning” consolation made me feel better, 
but my relief was short-lived. Here’s the question that bothered 
me most of all: if the goal of those months was to learn these 
important insights about customers, why did it take so long? 
How much of our effort contributed to the essential lessons we 
needed to learn? Could we have learned those lessons earlier if I 
hadn’t been so focused on making the product “better” by add- 
ing features and fixing bugs? 


VALUE VS. WASTE 


In other words, which of our efforts are value-creating and which 
are wasteful? This question is at the heart of the lean manufac- 
turing revolution; it is the first question any lean manufactur- 
ing adherent is trained to ask. Learning to see waste and then | 
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systematically eliminate it has allowed lean companies such as 
Toyota to dominate entire industries. In the world of software, 
the agile development methodologies I had practiced until that 
time had their origins in lean thinking. They were designed to 
eliminate waste too. | 

Yet those methods had led me down a road in which the ma- 
jority of my team’s efforts were wasted. Why? 

The answer came to me slowly over the subsequent years. Lean 
thinking defines value as providing benefit to the customer; any- 
thing else is waste. In a manufacturing business, customers don’t 
care how the product is assembled, only that it works correctly. 
But in a startup, who the customer is and what the customer 
might find valuable are unknown, part of the very uncertainty 
that is an essential part of the definition of a startup. I realized 
that as a startup, we needed a new definition of value. The real 
progress we had made at IMVU was what we had learned over 
those first months about what creates value for customers. 

Anything we had done during those months that did not 
contribute to our learning was a form of waste. Would it have 
been possible to learn the same cues with less effort? Clearly, 
the answer is yes. 

For one thing, think of all the debate and prioritization of ef- 
fort that went into features that customers would never discover. 
If we had shipped sooner, we could have avoided that waste. Also 
consider all the waste caused by our incorrect strategic assump- 
tions. I had built interoperability for more than a dozen different 
IM clients and networks. Was this really necessary to test our as- 
sumptions? Could we have gotten the same feedback from our 
customers with half as many networks? With only three? With 
only one? Since the customers of all IM networks found our prod- 
uct equally unattractive, the level of learning would have been the 
same, but our effort would have been dramatically less. 

Here's the thought that kept me up nights: did we have to 
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support any networks at all? Is it possible that we could have 
discovered how flawed our assumptions were without building 
anything? For example, what if we simply had offered custom- 
ers the opportunity to download the product from us solely on 
the basis of its proposed features before building anything? Re- 
member, almost no customers were willing to use our original 
product, so we wouldn't have had to do much apologizing when 
we failed to deliver. (Note that this is different from asking cus- 
tomers what they want. Most of the time customers don't know 
what they want in advance.) We could have conducted an ex- 
periment, offering customers the chance to try something and 
then measuring their behavior. 

Such thought experiments were extremely disturbing to 
me because they undermined my job description. As the head. 
of product development, I thought my job was to ensure the 
timely delivery of high-quality products and features. But if 
many of those features were a waste of time, what should I be 
doing instead? How could we avoid this waste? 

I’ve come to believe that learning is the essential unit of 
progress for startups. The effort that is not absolutely necessary 
for learning what customers want can be eliminated. I call this 
validated learning because it is always demonstrated by positive 
improvements in the startup’s core metrics. As we've seen, it's 
easy to kid yourself about what you think customers want. It's 
also easy to learn things that are completely irrelevant. Thus, 
validated learning is backed up by empirical data collected from 
real customers. 


WHERE DO YOU FIND VALIDATION? 


As I can attest, anybody who fails in a startup can claim that 
he or she has learned a lot from the experience. They can tell 
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a compelling story. In fact, in the story of IMVU so far, you 
might have noticed something missing. Despite my claims that 
we learned a lot in those early months, lessons that led to our 
eventual success, I haven't offered any evidence to back that up. 
In hindsight, it’s easy to make such claims and sound credible 
(and you'll see some evidence later in the book), but imagine us 
in IMVU’s early months trying to convince investors, employ- 
ees, family members, and most of all ourselves that we had not 
squandered our time and resources. What evidence did we have? 

Certainly our stories of failure were entertaining, and we had 
fascinating theories about what we had done wrong and what 
we needed to do to create a more successful product. However, 
the proof did not come until we put those theories into practice 
and built subsequent versions of the product that showed supe- 
rior results with actual customers. 

The next few months are where the true story of IMVU 
begins, not with our brilliant assumptions and strategies and 
whiteboard gamesmanship but with the hard work of discover- 
ing what customers really wanted and adjusting our product and 
strategy to meet those desires. We adopted the view that our job 
was to find a synthesis between our vision and what customers 
would accept; it wasn’t to capitulate to what customers thought 
they wanted or to tell customers what they ought to want. 

As we came to understand our customers better, we were 
able to improve our products. As we did that, the fundamen- 
tal metrics of our business changed. In the early days, despite 
our efforts to improve the product, our metrics were stubbornly 
flat. We treated each day’s customers as a new report card. We'd 
pay attention to the percentage of new customers who exhibited 
product behaviors such as downloading and buying our prod- 
uct. Each day, roughly the same number of customers would 
buy the product, and that number was pretty close to zero de- 

spite the many improvements. 
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However, once we pivoted away from the original strat- 
egy, things started to change. Aligned with a superior strat- 
egy, our product development efforts became magically more 
productive—not because we were working harder but because 
we were working smarter, aligned with our customers’ real 
needs. Positive changes in metrics became the quantitative vali- 
dation that our learning was real. This was critically important 
because we could show our stakeholders—employees, investors, 
and ourselves—that we were making genuine progress, not de- 
luding ourselves. It is also the right way to think about pro- 
ductivity in a startup: not in terms of how much stuff we are 
building but in terms of how much validated learning we're get- 
ting for our efforts.’ 

For example, in one early experiment, we changed our en- 
tire website, home page, and product registration flow to replace 
“avatar chat” with “3D instant messaging.” New customers were 
split automatically between these two versions of the site; half 
saw one, and half saw the other. We were able to measure the 
difference in behavior between the two groups. Not only were 
the people in the experimental group more likely to sign up for 
the product, they were more likely to become long-term paying 
- customers. 

We had plenty of failed experiments too. During one period 
in which we believed that customers weren't using the product 
because they didn’t understand its many benefits, we went so far 
as to pay customer service agents to act as virtual tour guides 
for new customers. Unfortunately, customers who got that 
VIP treatment were no more likely to become active or paying 
customers. 

Even after ditching the IM add-on strategy, it still took 
months to understand why it hadn’t worked. After our pivot and 
many failed experiments, we finally figured out this insight: cus- 
tomers wanted to use IMVU to make new friends online. Our 
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customers intuitively grasped something that we were slow to 
realize. All the existing social products online were centered on 
customers real-life identity. IMVU’s avatar technology, however, 
was uniquely well suited to help people get to know each other 
online without compromising safety or opening themselves up 
to identity theft. Once we formed this hypothesis, our experi- 
ments became much more likely to produce positive results. 
Whenever we would change the product to make it easier for 
people to find and keep new friends, we discovered that custom- 
ers were more likely to engage. This is true startup productivity: 
systematically figuring out the right things to build. 

These were just a few experiments among hundreds that we 
ran week in and week out as we started to learn which custom- 
ers would use the product and why. Each bit of knowledge we 
gathered suggested new experiments to run, which moved our 
metrics closer and closer to our goal. 


THE AUDACITY OF ZERO 


Despite IMVU’s early success, our gross numbers were still 
pretty small. Unfortunately, because of the traditional way 
businesses are evaluated, this is a dangerous situation. The 
irony is that it is often easier to raise money or acquire other 
resources when you have zero revenue, zero customers, and 
zero traction than when you have a small amount. Zero in- 
vites imagination, but small numbers invite questions about 
whether large numbers will ever materialize. Everyone knows 
(or thinks he or she knows) stories of products that achieved 
breakthrough success overnight. As long as nothing has been 
released and no data have been collected, it is'still possible to 
imagine overnight success in the future. Small numbers pour 
cold water on that hope. | 
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This phenomenon creates a brutal incentive: postpone get- 
ting any data until you are certain of success. Of course, as we'll 
see, such delays have the unfortunate effect of increasing the 
amount of wasted work, decreasing essential feedback, and dra- 
matically increasing the risk that a startup will build something 
nobody wants. 

However, releasing a product and hoping for the best is 
not a good plan either, because this incentive is real. When we 
launched IMVU, we were ignorant of this problem. Our earli- 
est investors and advisers thought it was quaint that we had a 
$300-per-month revenue plan at first. But after several months 
with our revenue hovering around $500 per month, some 
began to lose faith, as did some of our advisers, employees, and 
even spouses. In fact, at one point, some investors were seri- 
ously recommending that we pull the product out of the mar- 
ket and return to stealth mode. Fortunately, as we pivoted and 
experimented, incorporating what we learned into our product 
development and marketing efforts, our numbers started to 
improve. 

But not by much! On the one hand, we were lucky to see 
a growth pattern that started to look like the famous hockey 
stick graph. On the other hand, the graph went up only to a 
few thousand dollars per month. These early graphs, although 
promising, were not by themselves sufficient to combat the 
loss of faith caused by our early failure, and we lacked the lan- 
guage of validated learning to provide an alternative concept 
to rally around. We were quite fortunate that some of our 
early investors understood its importance and were willing to 
look beyond our small gross numbers to see the real progress 
we were making. (You'll see the exact same graphs they did in 
Chapter 7.) 

Thus, we can mitigate the waste that happens because of 
the audacity of zero with validated learning. What we needed 
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to demonstrate was that our product development efforts 
were leading us toward massive success without giving in to 
the temptation to fall back on vanity metrics and “success 
theater”—the work we do to make ourselves look successful. 
We could have tried marketing gimmicks, bought a Super 
Bowl ad, or tried flamboyant public relations (PR) as a way 
of juicing our gross numbers. That would have given investors 
the illusion of traction, but only for a short time. Eventually, 
the fundamentals of the business would win out and the PR 
bump would pass. Because we would have squandered pre- 
cious resources on theatrics instead of progress, we would have 
been in real trouble. 

Sixty million avatars later, IMVU is still going strong. Its leg- 
acy is not just a great product, an amazing team, and promising 
financial results but a whole new way of measuring the progress 
of startups. ; 


LESSONS BEYOND IMVU 


I have had many opportunities to teach the IMVU story as a 
business case ever since Stanford’s Graduate School of Business 
wrote an official study about IMVU’s early years.’ The case is 
now part of the entrepreneurship curriculum at several business 
schools, including Harvard Business School, where I serve as an 
entrepreneur in residence. I’ve also told these stories at countless 
workshops, lectures, and conferences. 

Every time I teach the IMVU story, students have an over- 
whelming temptation to focus on the tactics it illustrates: 
launching a low-quality early prototype, charging customers 
from day one, and using low-volume revenue targets as a way 
to drive accountability. These are useful techniques, but they are 
not the moral of the story. There are too many exceptions. Not 
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every kind of customer will accept a low-quality prototype, for 
example. If the students are more skeptical, they may argue that 
the techniques do not apply to their industry or situation, but 
work only because IMVU is a software company, a consumer 
Internet business, or a non-mission-critical application. 

None of these takeaways is especially useful. The Lean 
Startup is not a collection of individual tactics. It is a principled 
approach to new product development. The only way to make 
sense of its recommendations is to understand the underlying 
principles that make them work. As we'll see in later chapters, 
the Lean Startup model has been applied to a wide variety of 
businesses and industries: manufacturing, clean tech, restau- 
rants, and even laundry. The tactics from the IMVU story may 
or may not make sense in your particular business. 

Instead, the way forward is to learn to see every startup in 
any industry as a grand experiment. The question is not “Can 
this product be built?” In the modern economy, almost any 
product that can be imagined can be built. The more pertinent 
questions are “Should this product be built?” and “Can we build 
a sustainable business around this set of products and services?” 
To answer those questions, we need a method for systematically 
breaking down a business plan into its component parts and 
testing each part empirically. 

In other words, we need the scientific method. In the Lean 
Startup model, every product, every feature, every marketing 
campaign—everything a startup does—is understood to be an 
experiment designed to achieve validated learning. This experi- 
mental approach works across industries and sectors, as we'll see 


in Chapter 4. 
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come across many startups that are struggling to answer the follow- 
ing questions: Which customer opinions should we listen to, 
if any? How should we prioritize across the many features we 
could build? Which features are essential to the product’s success 
and which are ancillary? What can be changed safely, and what 
might anger customers? What might please today’s customers at 
the expense of tomorrow’s? What should we work on next? 
These are some of the questions teams struggle to answer if 
they have followed the “let’s just ship a product and see what 
happens” plan. I call this the “just do it” school of entrepreneur- 
ship after Nike’s famous slogan.’ Unfortunately, if the plan is 
to see what happens, a team is guaranteed to succeed—at see- 
ing what happens—but won't necessarily gain validated learn- 
ing. This is one of the most important lessons of the scientific 
method: if you cannot fail, you cannot learn. 


FROM ALCHEMY 10 SCIENCE 


The Lean Startup methodology reconceives a startup’s efforts as 
experiments that test its strategy to see which parts are brilliant 
and which are crazy. A true experiment follows the scientific 
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method. It begins with a clear hypothesis that makes predictions 
about what is supposed to happen. It then tests those predic- 
tions empirically. Just as scientific experimentation is informed 
by theory, startup experimentation is guided by the startup’s vi- 
sion. The goal of every startup experiment is to discover how to 
build a sustainable business around that vision. 


Think Big, Start Small 


Zappos is the world’s largest online shoe store, with annual gross 
sales in excess of $1 billion. It is known as one of the most suc- 
cessful, customer-friendly e-commerce businesses in the world, 
but it did not start that way. 

Founder Nick Swinmurn was frustrated because there was 
no central online site with a great selection of shoes. He envi- 
sioned a new and superior retail experience. Swinmurn could 
have waited a long time, insisting on testing his complete vision 
complete with warehouses, distribution partners, and the prom- 
ise of significant sales. Many early e-commerce pioneers did just 
that, including infamous dot-com failures such as Webvan and 
Pets.com. 

Instead, he started by running an experiment. His hypoth- 
esis was that customers were ready and willing to buy shoes on- 
line. To test it, he began by asking local shoe stores if he could 
take pictures of their inventory. In exchange for permission to 
take the pictures, he would post the pictures online and come 
back to buy the shoes at full price if a customer bought them 
online. 

Zappos began with a tiny, simple product. It was designed to 
answer one question above all: is there already sufficient demand 
for a superior online shopping experience for shoes? However, 
a well-designed startup experiment like the one Zappos began 
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with does more than test a single aspect of a business plan. In the 
course of testing this first assumption, many other assumptions 
were tested as well. To sell the shoes, Zappos had to interact 
with customers: taking payment, handling returns, and dealing 
with customer support. This is decidedly different from market 
research. If Zappos had relied on existing market research or 
conducted a survey, it could have asked what customers thought 
they wanted. By building a product instead, albeit a simple one, 
the company learned much more: 


1. It had more accurate data about customer demand be- 
cause it was observing real customer behavior, not asking - 
hypothetical questions. 

2. It put itself in a position to interact with real custom- 
ers and learn about their needs. For example, the busi- 
ness plan might call for discounted pricing, but how are 
customer perceptions of the product affected by the dis- 
counting strategy 

3. It allowed itself to be surprised when customers behaved 
in unexpected ways, revealing information Zappos might 
not have known to ask about. For example, what if cus- 
tomers returned the shoes? 


Zappos initial experiment provided a clear, quantifiable 
outcome: either a sufficient number of customers would buy 
the shoes or they would not. It also put the company in a po- 
sition to observe, interact with, and learn from real customers 
and partners. This qualitative learning is a necessary companion 
to quantitative testing. Although the early efforts were decid- 
edly small-scale, that did not prevent the huge Zappos vision 
from being realized. In fact, in 2009 Zappos was acquired by 
the e-commerce giant Amazon.com for a reported $1.2 billion.? — 
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For Long-Term Change, Experiment Immediately 


Caroline Barlerin is a director in the global social innovation 
division at Hewlett-Packard (HP), a multinational company 
with more than three hundred thousand employees and more 
than $100 billion in annual sales. Caroline, who leads global 
community involvement, is a social entrepreneur working to 
get more of HP’s employees to take advantage of the companys 
policy on volunteering. 

Corporate guidelines encourage every employee to spend up 
to four hours a month of company time volunteering in his or 
her community; that volunteer work could take the form of any 
philanthropic effort: painting fences, building houses, or even 
using pro bono or work-based skills outside the company. En- 
couraging the latter type of volunteering was Caroline's priority. 
Because of its talent and values, HP’s combined workforce has 
the potential to have a monumental positive impact. A designer 
could help a nonprofit with a new website design. A team of 
engineers could wire a school for Internet access. 

Caroline’s project is just beginning, and most employees do 
not know that this volunteering policy exists, and only a tiny 
fraction take advantage of it. Most of the volunteering has been 
of the low-impact variety, involving manual labor, even when 
the volunteers were highly trained experts. Barlerin’s vision is to 
take the hundreds of thousands of employees in the company 
and transform them into a force for social good. 

This is the kind of corporate initiative undertaken every day 
at companies around the world. It doesn't look like a startup 
by the conventional definition or what we see in the movies. 
On the surface it seems to be suited to traditional management 
and planning. However, I hope the discussion in Chapter 2 has 
prompted you to be a little suspicious. Here’s how we might 
analyze this project using the Lean Startup framework. 
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Caroline's project faces extreme uncertainty: there had never 
been a volunteer campaign of this magnitude at HP before. How 
confident should she be that she knows the real reasons people 
aren't volunteering? Most important, how much does she really 
know about how to change the behavior of hundreds of thou- 
sand people in more than 170 countries? Barlerin’s goal is to 
inspire her colleagues to make the world a better place. Looked 
at that way, her plan seems full of untested assumptions—and a 
lot of vision. 

In accordance with traditional management practices, Bar- 
lerin is spending time planning, getting buy-in from various 
departments and other managers, and preparing a road map of 
initiatives for the first eighteen months of her project. She also 
has a strong accountability framework with metrics for the im- 
pact her project should have on the company over the next four 
years. Like many entrepreneurs, she has a business plan that lays 
out her intentions nicely. Yet despite all that work, she is—so 
far—creating one-off wins and no closer to knowing if her vi- 
sion will be able to scale. 

One assumption, for example, might be that the company’s 
long-standing values included a commitment to improving the 
community but that recent economic trouble had resulted in an 
increased companywide strategic focus on short-term profitabil- 
ity. Perhaps longtime employees would feel a desire to reaffirm 
their values of giving back to the community by volunteering. 
A second assumption could be that they would find it more sat- 
isfying and therefore more sustainable to use their actual work- 
place skills in a volunteer capacity, which would have a greater 
impact on behalf of the organizations to which they donated 
their time. Also lurking within Caroline’s plans are many practi- 
cal assumptions about employees’ willingness to take the time to 
volunteer, their level of commitment and desire, and the way to 
best reach them with her message. 
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The Lean Startup model offers a way to test these hypoth- 
eses rigorously, immediately, and thoroughly. Strategic planning 
takes months to complete; these experiments could begin im- 
mediately. By starting small, Caroline could prevent a tremen- 
dous amount of waste down the road without compromising 
her overall vision. Here’s what it might look like if Caroline were 
to treat her project as an experiment. 


Break It Down 


The first step would be to break down the grand vision into 
its component parts. The two most important assumptions en- 
trepreneurs make are what I call the value hypothesis and the 
growth hypothesis. 

The value hypothesis tests whether a product or service really 
delivers value to customers once they are using it. What's a good 
indicator that employees find donating their time valuable? We 
could survey them to get their opinion, but that would not be 
very accurate because most people have a hard time assessing 
their feelings objectively. 

Experiments provide a more accurate gauge. What could we 
see in real time that would serve as a proxy for the value partici- 
pants were gaining from volunteering? We could find opportu- 
nities for a small number of employees to volunteer and then 
look at the retention rate of those employees. How many of 
them sign up to volunteer again? When an employee voluntarily 
invests their time and attention in this program, that is a strong 
indicator that they find it valuable. 

For the growth hypothesis, which tests how new customers 
will discover a product or service, we can do a similar analy- 
sis. Once the program is up and running, how will it spread 
among the employees, from initial early adopters to mass adop- 
tion throughout the company? A likely way this program could 
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expand is through viral growth. If that is true, the most impor- 
tant thing to measure is behavior: would the early participants 
actively spread the word to other employees? 

In this case, a simple experiment would involve taking a 
very small number—a dozen, perhaps—of existing long-term 
employees and providing an exceptional volunteer opportunity 
for them. Because Caroline’s hypothesis was that employees 
would be motivated by their desire to live up to HP’s histori- 
cal commitment to community service, the experiment would 
target employees who felt the greatest sense of disconnect be- 
tween their daily routine and the company’s expressed values. 
The point is not to find the average customer but to find early 
adopters: the customers who feel the need for the product most 
acutely. Those customers tend to be more forgiving of mistakes 
and are especially eager to give feedback. 

Next, using a technique I call the concierge minimum vi- 
able product (described in detail in Chapter 6), Caroline could 
make sure the first few participants had an experience that was 
as good as she could make it, completely aligned with her vi- 
sion. Unlike in a focus group, her goal would be to measure 
what the customers actually did. For example, how many of : 
the first volunteers actually complete their volunteer assign- 
ments? How many volunteer a second time? How many are 
willing to recruit a colleague to participate in a subsequent 
volunteer activity? 

Additional experiments can expand on this early feedback 
and learning. For example, if the growth model requires that a 
certain percentage of participants share their experiences with 
colleagues and encourage their participation, the degree to 
which that takes place can be tested even with a very small sam- 
ple of people. If ten people complete the first experiment, how 
many do we expect to volunteer again? If they are asked to re- 
cruit a colleague, how many do we expect will do so? Remember 
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that these are supposed to be the kinds-of early adopters with 
the most to gain from the program. 

Put another way, what if all ten early adopters decline to 
volunteer again? That would be a highly significant—and very 
negative—result. If the numbers from such early experiments 
don’t look promising, there is clearly a problem with the strategy. 
That doesn’t mean it’s time to give up; on the contrary, it means 
it’s time to get some immediate qualitative feedback about how 
to improve the program. Here’s where this kind of experimenta- 
" tion has an advantage over traditional market research. We dont 
have to commission a survey or find new people to interview. 
We already have a cohort of people to talk to as well as knowl- 
edge about their actual behavior: the participants in the initial 
experiment. 

This entire experiment could be conducted in a matter of 
weeks, less than one-tenth the time of the traditional strategic 
planning process. Also, it can happen in parallel with strategic 
planning while the plan is still being formulated. Even when 
experiments produce a negative result, those failures prove in- 
structive and can influence the strategy. For example, what if 
no volunteers can be found who are experiencing the conflict 
of values within the organization that was such an impor- 
tant assumption in the business plan? If so, congratulations: 
it’s time to pivot (a concept that is explored in more detail in 


Chapter 8).° 


AN EXPERIMENT 15 A PRODUCT 


In the Lean Startup model, an experiment is more than just a 
theoretical inquiry; it is also a first product. If this or any other 
experiment is successful, it allows the manager to get started with 
his or her campaign: enlisting early adopters, adding employees 
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to each further experiment or iteration, and eventually starting 
to build a product. By the time that product is ready to be dis- 
tributed widely, it will already have established customers. It will 
have solved real problems and offer detailed specifications for 
what needs to be built. Unlike a traditional strategic planning 
or market research process, this specification will be rooted in 
feedback on what is working today rather than in anticipation 
of what might work tomorrow. 

To see this in action, consider an example from Kodak. Ko- 
dak’s history is bound up with cameras and film, but today it ' 
also operates a substantial online business called Kodak Gallery. 
Mark Cook is Kodak Gallery's vice president of products, and 
he is working to change Kodak Gallery’s culture of development 
to embrace experimentation. 

Mark explained, “Traditionally, the product managez says, 
‘I just want this.’ In response, the engineer says, ‘I’m going to 
build it.’ Instead, I try to push my team to first answer four 
questions: 


1. Do consumers recognize that they have the problem you 
are trying to solve? 

2. If there was a solution, would they buy it? 

3. Would they buy it from us? 


4, Can we build a solution for that problem?” 


The common tendency of product development is to skip 
straight to the fourth question and build a solution before con- 
firming that customers have the problem. For example, Kodak 
Gallery offered wedding cards with gilded text and graphics on 
its site. Those designs were popular with customers who were 
getting married, and so the team redesigned the cards to be used 
at other special occasions, such as for holidays. The market re- 
search and design process indicated that customers would like 
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the new cards, and that finding justified the significant effort 
that went into creating them. 

Days before the launch, the team realized the aids were too 
difficult to understand from their depiction on the website; peo- 
ple couldn’t see how beautiful they were. They were also hard to 
produce. Cook realized that they had done the work backward. 
He explained, “Until we could figure out how to sell and make 
the product, it wasn’t worth spending any engineering time on.” 

Learning from that experience, Cook took a different ap- 
proach when he led his team through the development of a new 
set of features for a product that makes it easier to share photos 
taken at an event. They believed that an online “event album” 
would provide a way for people who attended a wedding, a con- 
ference, or another gathering to share photos with other attend- 
ees. Unlike other online photo sharing services, Kodak Gallery's 
event album would have strong privacy controls, assuring that 
the photos would be shared only with people who attended the 
same event. 

In a break with the past, Cook led the group through a pro- 
cess of identifying risks and assumptions before building any- 
thing and then testing those assumptions experimentally. 

There were two main hypotheses underlying the proposed 
event album: 


1. The team assumed that customers would want to create 
the albums in the first place. 

2. It assumed that event participants would upload photos 
to event albums created by friends or colleagues. 


The Kodak Gallery team built a simple prototype of the 
event album. It lacked many features—so many, in fact, that the 
team was reluctant to show it to customers. However, even at 
that early stage, allowing customers to use the prototype helped 
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the team refute their hypotheses. First, creating an album was 
not as easy as the team had predicted; none of the early custom- 
ers were able to create one. Further, customers complained that 
the early product version lacked essential features. 

Those negative results demoralized the team. The usability 
problems frustrated them, as did customer complains about 
missing features, many of which matched the original road map. 
Cook explained that even though the product was missing fea- 
tures, the project was not a failure. The initial product—flaws 
and all—confirmed that users did have the desire to create event 
albums, which was extremely valuable information. Where cus- 
tomers complained about missing features, this suggested that 
the team was on the right track. The team now had early evi- 
dence that those features were in fact important. What about 
features that were on the road map but that customers didn’t 
complain about? Maybe those features weren't as important as 
they initially seemed. 

Through a beta launch the team continued to learn and it- 
erate. While the early users were enthusiastic and the numbers 
were promising, the team made a major discovery. Through the 
use of online surveying tool KISSinsights, the team learned that 
many customers wanted to be able to arrange the order of pic- 
tures before they would invite others to contribute. Knowing 
they weren't ready to launch, Cook held off his division's general 
manager by explaining how iterating and experimenting before 
beginning the marketing campaign would yield far better re- 
sults. In a world where marketing launch dates were often set 
months in advance, waiting until the team had really solved the 
problem was a break from the past. 

This process represented a dramatic change for Kodak Gal- 
lery; employees were used to being measured on their progress at 
completing tasks. As Cook says, “Success is not delivering a fea- 
ture; success. is learning how to solve the customer’s problem.”4 
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THE VILLAGE LAUNDRY SERVICE 


In India, due to the cost of a washing machine, less than seven 
percent of the population have one in their homes. Most people 
either hand wash their clothing at home or pay a Dhobi to do 
it for them. Dhobis take the clothes to the nearest river, wash 
them in the river water, bang them against rocks to get them 
clean, and hang them to dry, which takes two to seven days. The 
result? Clothes are returned in about ten days and are probably 
not that clean. 

Akshay Mehra had been working at Procter & Gamble Sin- 
gapore for eight years when he sensed an opportunity. As the 
brand manager of the Tide and Pantene brands for India and 
ASEAN countries, he thought he could make laundry services 
available to people who previously could not afford them. Re- 
turning to India, Akshay joined the Village Laundry Services 
(VLS), created by Innosight Ventures. VLS began a series of ex- 
periments to test its business assumptions. 

For their first experiment, VLS mounted a consumer-grade 
laundry machine on the back of a pickup truck parked on a 
street corner in Bangalore. The experiment cost less than $8,000 
and had the simple goal of proving that people would hand over 
their laundry and pay to have it cleaned. The entrepreneurs did 
not clean the laundry on the truck, which was more for market- 
ing and show, but took it off-site to be cleaned and a it 
back to their customers by the end of the day. 

The VLS team continued the experiment for a week, ves 
the truck on different street corners, digging deeper to discover 
all they could about their potential customers. They wanted to 
know how they could encourage people to come to the truck. 
Did cleaning speed matter? Was cleanliness a concern? What 
were people asking for when they left their laundry with them? 
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They discovered that customers were happy to give them their 
laundry to clean. However, those customers were suspicious of 
the washing machine mounted on the back of the truck, con- 
cerned that VLS would take their laundry and run. To address 
that concern, VLS created a slightly more substantial mobile 
cart that looked more like a kiosk. 

_ VLS also experimented with parking the carts in front of a 
local minimarket chain. Further iterations helped VLS figure 
out which services people were most interested in and what 
price they were willing to pay. They discovered that customers 
often wanted their clothes ironed and were willing to pay dou- 
ble the price to get their laundry back in four hours rather than 
twenty-four hours. 

As a result of those early experiments, VLS created an end 
product that was a three-foot by four-foot mobile kiosk that in- 
cluded an energy-efficient, consumer-grade washing machine, a 
dryer, and an extra-long extension cord. The kiosk used Western 
detergents and was supplied daily with fresh clean water deliv-— 
ered by VLS. 

Since then, the Village Laundry Service has grown substan- 
tially, with fourteen locations operational in Bangalore, Mysore, 
and Mumbai. As CEO Akshay Mehra shared with me, “We 
have serviced 116,000 kgs. in 2010 (vs. 30,600 kg. in 2009). 
And almost 60 percent of the business is coming from repeat 
customers. We have serviced more than 10,000 customers in the 
past year alone across all the outlets.”° 


A LEAN STARTUP IN GOVERNMENT? 


On July 21, 2010, President Obama signed the Dodd—Frank 
Wall Street Reform and Consumer Protection Act into law. One 
of its landmark provisions created a new federal agency, the 
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Consumer Federal Protection Bureau (CFPB). This agency is 
tasked with protecting American citizens from predatory lend- 
ing by financial services companies such as credit card compa- 
nies, student lenders, and payday loan offices. The plan calls for 
it to accomplish this by setting up a call center where trained 
case workers will field calls directly from the public. 

Left to its own devices, a new government agency would 
probably hire a large staff with a large budget to develop a plan 
that is expensive and time-consuming. However, the CFPB is 
considering doing things differently. Despite its $500 million 
budget and high-profile origins, the CPFB is really a startup. 

President Obama tasked his chief technology officer, Aneesh 
Chopra, with collecting ideas for how to set up the new startup 
agency, and that is how I came to be involved. On one of Cho- 
pra’s visits to Silicon Valley, he invited a number of entrepreneurs 
to make suggestions for ways to cultivate a startup mentality 
in the new agency. In particular, his focus was on leveraging 
technology and innovation to make the agency more efficient, 
cost-effective, and thorough. 

My suggestion was drawn straight from the principles of 
this chapter: treat the CFPB as an experiment, identify the ele- 
ments of the plan that are assumptions rather than facts, and 
figure out ways to test them. Using these insights, we could 
build a minimum viable product and have the agency up and 
running—on a micro scale—long before the official plan was 
set in motion. 

The number one assumption underlying the current plan is 
that once Americans know they can call the CFPB for help with 
financial fraud and abuse, there will be a significant volume of 
citizens who do that. This sounds reasonable, as it is based on 
market research about the amount of fraud that affects Americans 
each year. However, despite all that research, it is still an assump- 
tion. If the actual call volume differs markedly from that in the 
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plan, it will require significant revision. What if Americans who 
are subjected to financial abuse don’t view themselves as victims 
and therefore don’t seek help? What if they have very different 
notions of what problems are important? What if they call the 
agency seeking help for problems that are outside its purview? 

Once the agency is up and running with a $500 million bud- 
get and a correspondingly large staff, altering the plan will be 
expensive and time-consuming, but why wait to get feedback? 
To start experimenting immediately, the agency could start with 
the creation of a simple hotline number, using one of the new 
breed of low-cost and fast setup platforms such as Twilio. With a 
few hours’ work, they could add simple voice prompts, offering 
callers a menu of financial problems to choose from. In the first 
version, the prompts could be drawn straight from the existing 
research. Instead of a caseworker on the line, each prompt could 
offer the caller useful information about how to solve her or his 
problem. | 

Instead of marketing this hotline to the whole country, the 
agency could run the experiment in a much more limited way: 
start with a small geographic area, perhaps as small as a few city 
blocks, and instead of paying for expensive television or radio 
advertising to let people know about the service, use highly 
targeted advertising. Flyers on billboards, newspaper advertise- 
ments to those blocks, or specially targeted online ads would be 
a good start. Since the target area is so small, they could afford 
to pay a premium to create a high level of awareness in the target 
zone. The total cost would remain quite small. 

As a comprehensive solution to the problem of financial 
abuse, this minimum viable product is not very good compared 
with what a $500 million agency could accomplish. But it is 
also not very expensive. This product could be built in a mat- 
ter of days or weeks, and the whole experiment probably would 
cost only a few thousand dollars. 
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What we would learn from this experiment would be in- 
valuable. On the basis of the selections of those first callers, 
the agency could immediately start to get a sense of what kinds 
of problems Americans believe they have, not just what they 
“should” have. The agency could begin to test marketing mes- 
sages: What motivates people to call? It could start to extrapo- 
late real-world trends: What percentage of people in the target 
area actually call? The extrapolation would not be perfect, but it 
would establish a baseline behavior that would be far more ac- 
curate than market research. 

Most important, this product would serve as a seed that 
could germinate into a much more elaborate service. With this 
beginning, the agency could engage in a continuous process of 
improvement, slowly but surely adding more and better solu- 
tions. Eventually, it would staff the hotline with caseworkers, 
perhaps at first addressing only one category of problems, to 
give the caseworkers the best chance of success. By the time the 
official plan was ready for implementation, this early service 
could serve as a real-world template. 

The CFPB is just getting started, but already they are show- 
ing signs of following an experimental approach. For example, 
instead of doing a geographically limited rollout, they are seg- 
menting their first products by use case. They have established 
a preliminary order of financial products to provide consumer 
services for, with credit cards coming first. As their first experi- 
ment unfolds, they will have the opportunity to closely monitor 
all of the other complaints and consumer feedback they receive. 
This data will influence the depth, breadth, and sequence of fu- 
ture offerings. 

As David Forrest, the CFPB’s chief technology officer, told 
me, “Our goal is to give American citizens an easy way to tell us 
about the problems they see out there in the consumer financial 
marketplace. We have an opportunity to closely monitor what 
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the public is telling us and react to new information. Markets 
change all the time and our job is to change with them.”® 


O00 


The entrepreneurs and managers profiled in this book are smart, 
capable, and extremely results-oriented. In many cases, they are 
in the midst of building an organization in a way consistent 
with the best practices of current: management thinking. They 
face the same challenges in both the public and private sectors, 
regardless of industry. As we've seen, even the seasoned. manag- 
ers and executives at the world’s best-run companies struggle to 
consistently develop and launch innovative new products. 

Their challenge is to overcome the prevailing management 
thinking that puts its faith in well-researched plans. Remember, 
planning is a tool that only works in the presence of a long and 
stable operating history. And yet, do any of us feel that the world 
around us is getting more and more stable every day? Changing 
such a mind-set is hard but critical to startup success. My hope 
is that this book will help managers and entrepreneurs make this 
change. 


Part Iwo 


STEER 


How Vision Leads to oteering 





At its heart, a startup is a catalyst that transforms ideas into 
products. As customers interact with those products, they gen- 
erate feedback and data. The feedback is both qualitative (such 
as what they like and dont’ like) and quantitative (such as how 
many people use it and find it valuable). As we saw in Part One, 
the products a startup builds are really experiments; the learn- 
ing about how to build a sustainable business is the outcome of 
those experiments. For startups, that information is much more 
important than dollars, awards, or mentions in the press, be- 
cause it can influence and reshape the next set of ideas. 

We can visualize this three-step process with this simple 
diagram: 


BUILD-MEASURE-LEARN FEEDBACK LOOP 





Minimize TOTAL time through the loop 
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This Build-Measure-Learn feedback loop is at the core of.the 


Lean Startup model. In Part Two, we will examine it in great 
detail. 

Many people have professional training that emphasizes one 
element of this feedback loop. For engineers, it’s learning to 
build things as efficiently as possible. Some managers are ex- 
perts at strategizing and learning at the whiteboard. Plenty of 
entrepreneurs focus their energies on the individual nouns: hav- 
ing the best product idea or the best-designed initial product or 
obsessing over data and metrics. The truth is that none of these 
activities by itself is of paramount importance. Instead, we need 
to focus our energies on minimizing the total time through this 
feedback loop. This is the essence of steering a startup and is the 
subject of Part Two. We will walk through a complete turn of 
the Build-Measure-Learn feedback loop, anit each of the 
components in detail. 

The purpose of Part One was to ee the importance of 
learning as the measure of progress for a startup. As I hope is 
evident by now, by focusing our energies on validated learning, 
we can avoid much of the waste that plagues startups today. As 
in lean manufacturing, learning where and when to invest en- 
ergy results in saving time and money. 

To apply the scientific method to a startup, we need to 
identify which hypotheses to test. I call the riskiest elements 
of a startup’s plan, the parts on which everything depends, 
leap-of-faith assumptions. The two most important assumptions 
are the value hypothesis and the growth hypothesis. These give 
rise to tuning variables that control a startup’s engine of growth. 
Each iteration of a startup is an attempt to rev this engine to see 
if it will turn. Once it is running, the process repeats, shifting 
into higher and higher gears. 

Once clear on these leap-of-faith assumptions, the first step is 
to enter the Build phase as quickly as possible with a minimum 
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viable product (MVP). The MVP is that version of the product 
that enables a full turn of the Build-Measure-Learn loop with 
a minimum amount of effort and the least amount of develop- 
ment time. The minimum viable product lacks many features 
that may prove essential later on. However, in some ways, creat- 
ing a MVP requires extra work: we must be able to measure its 
impact. For example, it is inadequate to build a prototype that is 
evaluated solely for internal quality by engineers and designers. 
We also need to get it in front of potential customers to gauge 
their reactions. We may even need to try selling them the proto- 
type, as we'll soon see. 

When we enter the Measure phase, the biggest challenge will 
be determining whether the product development efforts are 
leading to real progress. Remember, if we're building something 
that nobody wants, it doesn’t much matter if we're doing it on 
time and on budget. The method I recommend is called inno- 
vation accounting, a quantitative approach that allows us to see 
whether our engine-tuning efforts are bearing fruit. It also allows 
us to create earning milestones, which are an alternative to tradi- 
tional business and product milestones. Learning milestones are 
useful for entrepreneurs as a way of assessing their progress accu- 
rately and objectively; they are also invaluable to managers and 
investors who must hold entrepreneurs accountable. However, 
not all metrics are created equal, and in Chapter 7 I'll clarify 
the danger of vanity metrics in contrast to the nuts-and-bolts 
usefulness of actionable metrics, which help to analyze customer 
behavior in ways that support innovation accounting. 

Finally, and most important, there's the pivot. Upon com- 
pleting the Build-Measure-Learn loop, we confront the most 
difficult question any entrepreneur faces: whether to pivot the 
original strategy or persevere. If we've discovered that one of our 
hypotheses is false, it is time to make a major change to a new 
strategic hypothesis. 


18 THE LEAN STARTUP 


The Lean Startup method builds capital-efficient com- 
panies because it allows startups to recognize that it’s time to 
pivot sooner, creating less waste of time and money. Although 
we write the feedback loop as Build-Measure-Learn because the 
activities happen in that order, our planning really works in the 
reverse order: we figure out what we need to learn, use innova- 
tion accounting to figure out what we need to measure to know 
if we are gaining validated learning, and then figure out what 
product we need to build to run that experiment and get that 
measurement. All of the techniques in Part Two are designed to 
minimize the total time through the Build-Measure-Learn feed- 


back loop. 


LEAP 


n 2004, three college sophomores arrived in Silicon Valley with 

their fledgling college social network. It was live on a handful 
of college campuses. It was not the market-leading social net- 
work or even the first college social network; other companies 
had launched sooner and with more features. With 150,000 
registered users, it made very little revenue, yet that summer 
they raised their first $500,000 in venture capital. Less than a 
year later, they raised an additional $12.7 million. 

Of course, by now you've guessed that these three college 
sophomores were Mark Zuckerberg, Dustin Moskovitz, and 
Chris Hughes of Facebook. Their story is now world famous. 
Many things about it are remarkable, but I'd like to focus on 
only one: how Facebook was able to raise so much money when 
its actual usage was so small.! 

By all accounts, what impressed investors the most were two 
facts about Facebook's early growth. The first fact was the raw 
amount of time Facebook's active users spent on the site. More 
than half of the users came back to the site every single day.” 
This is an example of how a company can validate its value hy- 
pothesis—that customers find the product valuable. The second 
impressive thing about Facebook's early traction was the rate 
at which it had taken over its first few college campuses. The 


B0 THE LEAN STARTUP 


rate of growth was staggering: Facebook launched on Febru- 
ary 4, 2004, and by the end of that month almost three-quarters 
of Harvard’s undergraduates were using it, without a dollar of 
marketing or advertising having been spent. In other words, 
Facebook also had validated its growth hypothesis. These two 
hypotheses represent two of the most important /eap-of-faith 
questions any new startup faces.” = 

At the time, I heard many people criticize Facebook's early 
investors, claiming that Facebook had “no business model” and 
only modest revenues relative to the valuation offered by its in- 
vestors. They saw in Facebook a return to the excesses of the 
dot-com era, when companies with little revenue raised massive 
amounts of cash to pursue a strategy of “attracting eyeballs” and 
“getting big fast.” Many dot-com-era startups planned to make 
money later by selling the eyeballs they had bought to other ad- 
vertisers. In truth, those dot-com failures were little more than 
middlemen, effectively paying money to acquire customers’ at- 
tention and then planning to resell it to others. Facebook was 
different, because it employed a different engine of growth. It 
paid nothing for customer acquisition, and its high engagement 
meant that it was accumulating massive amounts of customer 
attention every day. There was never any question that atten- 
tion would be valuable to advertisers; the only question was how 
much they would pay. | 

Many entrepreneurs are attempting to build the next Face- 
book, yet when they try to apply the lessons of Facebook and 
other famous startup success stories, they quickly get confused. 
Is the lesson of Facebook that startups should not charge cus- 
tomers money in the early days? Or is it that startups should 
never spend money on marketing? These questions cannot be 
answered in the abstract; there are an almost infinite number of 
counterexamples for any technique. Instead, as we saw in Part 
One, startups need to conduct experiments that help determine 
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what techniques will work in their unique circumstances. For 
startups, the role of strategy is to help figure out the right ques- 
tions to ask. 


STRATEGY 1S BASED ON ASSUMPTIONS 


Every business plan begins with a set of assumptions. It lays out 
a strategy that takes those assumptions as a given and proceeds 
to show how to achieve the company’s vision. Because the as- 
sumptions haven't been proved to be true (they are assumptions, 
after all) and in fact are often erroneous, the goal of a startup's 
early efforts should be to test them as quickly as possible. 

What traditional business strategy excels at is helping man- 
agers identify clearly what assumptions are being made in a 
particular business. The first challenge for an entrepreneur is to 
build an organization that can test these assumptions systemati- 
cally. The second challenge, as in all entrepreneurial situations, 
is to perform that rigorous testing without losing sight of the 
company’s overall vision. 

Many assumptions in a typical business plan are unexcep- 
tional. These are well-established facts drawn from past indus- 
try experience or straightforward deductions. In Facebook's 
case, it was clear that advertisers would pay for customers’ at- 
tention. Hidden among these mundane details are a handful of 
assumptions that require more courage to state—in the present 
tense—with a straight face: we assume that customers have a 
significant desire to use a product like ours, or we assume that 
supermarkets will carry our product. Acting as if these assump- 
tions are true is a classic entrepreneur superpower. They are 
called leaps of faith precisely because the success of the entire 
venture rests on them. If they are true, tremendous opportunity 
awaits. If they are false, the startup risks total failure. 
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Most leaps of faith take the form of an argument by analogy. 
For example, one business plan I remember argued as follows: 
“Just as the development of progressive image loading allowed 
the widespread use of the World Wide Web over dial-up, so too 
our progressive rendering technology will allow our product to 
run on low-end personal computers.” You probably have no idea 
what progressive image loading or rendering is, and it doesn’t 
much matter. But you know the argument (perhaps you've even 
used it): 


Previous technology X was used to win market Y because 
of attribute Z. We have a new technology X2 that will en- 
able us to win market Y2 because we too have attribute Z. 


The problem with analogies like this is that they obscure 
the true leap of faith. That is their goal: to make the business 
seem less risky. They are used to persuade investors, employees, 
or partners to sign on. Most entrepreneurs would cringe to see 
their leap of faith written this way: 


Large numbers of people already wanted access to the 
World Wide Web. They knew what it was, they could 
afford it, but they could not get access to it because the 
time it took to load images was too long. When progres- 
sive image loading was introduced, it allowed people to 
get onto the World Wide Web and tell their friends about 
it. Thus, company X won market Y. 

Similarly, there is already a large number of potential 
customers who want access to our product right now. 
They know they want it, they can afford it, but they can- 
not access it because the rendering is too slow. When we 
debut our product with progressive rendering technology, 
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they will flock to our software and tell their friends, and 
we will win market Y2. 


There are several things to notice in this revised statement. 
First, it’s important to identify the facts clearly. Is it really true 
that progressive image loading caused the adoption of the World 
Wide Web, or was this just one factor among many? More im- 
portant, is it really true that there are large numbers of potential 
customers out there who want our solution right now? The ear- 
lier analogy was designed to convince stakeholders that a reason- 
able first step is to build the new startup’s technology and see if 
customers will use it. The restated approach should make clear 
that what is needed is to do some empirical testing first: let's 
make sure that there really are hungry customers out there eager 
to embrace our new technology. 


Analogs and Antilogs 


There is nothing intrinsically wrong with basing strategy on 
comparisons to other companies and industries. In fact, that 
approach can help you discover assumptions that are not 
really leaps of faith. For example, the venture capitalist Randy 
Komisar, whose book Getting to Plan B discussed the concept of 
leaps of faith in great detail, uses a framework of “analogs” and 
“antilogs” to plot strategy. 

He explains the analog-antilog concept by using the iPod as 
an example. “If you were looking for analogs, you would have 
to look at the Walkman,” he says. “It solved a critical question 
that Steve Jobs never had to ask himself: Will people listen to 
music in a public place using earphones? We think of that as 
a nonsense question today, but it is fundamental. When Sony 
asked the question, they did not have the answer. Steve Jobs 
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had [the answer] in the analog [version]” Sony’s Walkman was 
the analog. Jobs then had to face the fact that although people 
were willing to download music, they were not willing to pay 
for it. “Napster was an antilog. That antilog had to lead him to 
address his business in a particular way,” Komisar says. “Out of 
these analogs and antilogs come a series of unique, unanswered 
questions. Those are leaps of faith that I, as an entrepreneur, 
am taking if I go through with this business venture. They are 
going to make or break my business. In the iPod business, one 
of those leaps of faith was that people would pay for music.” 
Of course that leap of faith turned out to be correct.* 


Beyond “The Right Place at the Right Time” 


There are any number of famous entrepreneurs who made mil- 
lions because they seemed to be in the right place at the right 
time. However, for every successful entrepreneur who was in the 
right place in the right time, there are many more who were 
there, too, in that right place at the right time but still man- 
aged to fail. Henry Ford was joined by nearly five hundred other 
entrepreneurs in the early twentieth century. Imagine being an 
automobile entrepreneur, trained in state-of-the-art engineer- 
ing, on the ground floor of one of the biggest market oppor- 
tunities in history. Yet the vast majority managed to make no 
money at all.5 We saw the same phenomenon with Facebook, 
which faced early competition from other college-based social 
networks whose head start proved irrelevant. 

What differentiates the success stories from the failures is 
that the successful entrepreneurs had the foresight, the ability, 
and the tools to discover which parts of their plans were work- 
ing brilliantly and which were misguided, and adapt their strate- 
gies accordingly. 


Leap 80 


Value and Growth 


As we saw in the Facebook story, two leaps of faith stand 
above all others: the value creation hypothesis and the growth 
hypothesis. The first step in understanding a new product or 
service is to figure out if it is fundamentally value-creating or 
value-destroying. I use the language of economics in referring to 
value rather than profit, because entrepreneurs include people 
who start not-for-profit social ventures, those in public sector 
startups, and internal change agents who do not judge their 
success by profit alone. Even more confusing, there are many 
organizations that are wildly profitable in the short term but 
ultimately value-destroying, such as the organizers of Ponzi 
schemes, and fraudulent or misguided companies (e.g., Enron 
and Lehman Brothers). 

A similar thing is true for growth. As with value, it’s essen- 
tial that entrepreneurs understand the reasons behind a startup’s 
growth. There are many value-destroying kinds of growth that 
should be avoided. An example would be a business that grows 
through continuous fund-raising from investors and lots of paid 
advertising but does not develop a value-creating product. 

Such businesses are engaged in what I call success theater, 
using the appearance of growth to make it seem that they are suc- 
cessful. One of the goals of innovation accounting, which is dis- 
cussed in depth in Chapter 7, is to help differentiate these false 
startups from true innovators. Traditional accounting judges new 
ventures by the same standards it uses for established companies, 
but these indications are not reliable predictors of a startup’s fu- 
ture prospects. Consider companies such as Amazon.com that 
racked up huge losses on their way to breakthrough success. 

Like its traditional counterpart, innovation accounting re- 
quires that a startup have and maintain a quantitative financial 
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model that can be used to evaluate progress rigorously. How- 
ever, in a startup’s earliest days, there is not enough data to 
make an informed guess about what this model might look 
like. A startup’s earliest strategic plans are likely to be hunch- or 
intuition-guided, and that is a good thing. To translate those 
instincts into data, entrepreneurs must, in Steve Blank’s famous 
phrase, “get out of the building” and start learning. 


GENCHI GEMBUTSU 


The importance of basing strategic decisions on firsthand un- 
derstanding of customers is one of the core principles that un- 
derlies the Toyota Production System. At Toyota, this goes by 
the Japanese term genchi gembutsu, which is one of the most 
important phrases in the lean manufacturing vocabulary. In 
English, it is usually translated as a directive to “go and see for 
yourself” so that business decisions can be based on deep first- 
hand knowledge. Jeffrey Liker, who has extensively documented 
the “Toyota Way,” explains it this way: 


In my Toyota interviews, when I asked what distinguishes 
the Toyota Way from other management approaches, 
the most common first response was genchi gembutsu— 
whether I was in manufacturing, product development, 
sales, distribution, or public affairs. You cannot be sure 
you really understand any part of any business problem 
unless you go and see for yourself firsthand. It is unac- 
ceptable to take anything for granted or to rely on the 
reports of others.° 


To demonstrate, take a look at the development of Toyota's 
Sienna minivan for the 2004 model year. At Toyota, the manager 


leap 8! 


responsible for the design and development of a new model is 
called the chief engineer, a cross-functional leader who oversees 
the entire process from concept to production. The 2004 Sienna 
was assigned to Yuji Yokoya, who had very little experience in 
North America, which was the Sienna’s primary market. To fig- 
ure out how to improve the minivan, he proposed an audacious 
entrepreneurial undertaking: a road trip spanning all fifty U.S. 
states, all thirteen provinces and territories of Canada, and all 
parts of Mexico. In all, he logged more than 53,000 miles of 
driving. In small towns and large cities, Yokoya would rent a 
current-model Sienna, driving it in addition to talking to and 
observing real customers. From those firsthand observations, 
Yokoya was able to start testing his critical assumptions about 
what North American consumers wanted in a minivan. 

It is common to think of selling to consumers as easier than 
selling to enterprises, because customers lack the complexity 
of multiple departments and different people playing different 
roles in the purchasing process. Yokoya discovered this was un- 
true for his customers: “The parents and grandparents may own 
the minivan. But it’s the kids who rule it. It’s the kids who oc- 
cupy the rear two-thirds of the vehicle. And it’s the kids who are 
the most critical—and the most appreciative of their environ- 
ment. If I learned anything in my travels, it was the new Sienna 
would need kid appeal.” Identifying these assumptions helped 
guide the car’s development. For example, Yokoya spent an un- 
usual amount of the Sienna’s development budget on internal 
comfort features, which are critical to a long-distance family 
road trip (such trips are much more common in America than 
in Japan). 

The results were impressive, boosting the Siennas market. 
share dramatically. The 2004 model’s sales were 60 percent 
higher than those in 2003. Of course, a product like the Si- 
enna is a classic sustaining innovation, the kind that the world’s 
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best-managed established companies, such as Toyota, excel at. 
Entrepreneurs face a different set of challenges because they op- 
erate with much higher uncertainty. While a company work- 
ing on a sustaining innovation knows enough about who and 
where their customers are to use genchi gembutsu to discover 
what customers want, startups’ early contact with potential cus- 
tomers merely reveals what assumptions require the most urgent 
testing. 


GET OUT OF THE BUILDING 


Numbers tell a compelling story, but I always remind entrepre- 
neurs that metrics are people, too. No matter how many inter- 
mediaries lie between a company and its customers, at the end 
of the day, customers are breathing, thinking, buying individu- 
als. Their behavior is measurable and changeable. Even when 
one is selling to large institutions, as in a business-to-business 
model, it helps to remember that those businesses are made up 
of individuals. All successful sales models depend on breaking 
down the monolithic view of organizations into the disparate 
people that make them up. 

As Steve Blank has been teaching entrepreneurs for years, the 
facts that we need to gather about customers, markets, suppliers, 
and channels exist only “outside the building.” Startups need 
extensive contact with potential customers to understand them, 
so get out of your chair and get to know them. 

The first step in this process is to confirm that your 
leap-of-faith questions are based in reality, that the customer 
has a significant problem worth solving. When Scott Cook 
conceived Intuit in 1982, he had a vision—at that time quite 
radical—that someday consumers would use personal comput- 


ers to pay bills and keep track of expenses. When Cook left his 
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consulting job to take the entrepreneurial plunge, he didn’t start 
with stacks of market research or in-depth analysis at the white- 
board. Instead, he picked up two phone books: one for Palo 
Alto, California, where he was living at the time, and the other 
for Winnetka, Illinois. 

Calling people at random, he inquired if he could ask them a 
few questions about the way they managed their finances. Those 
early conversations were designed to answer this leap-of-faith 
question: do people find it frustrating to pay bills by hand? It 
turned out that they did, and this early validation gave Cook the 
confirmation he needed to get started on a solution.’ 

Those early conversations did not delve into the product 
features of a proposed solution; that attempt would have been 
foolish. The average consumers at that time were. not conver- 
sant enough with personal computers to have an opinion about 
whether they'd want to use them in a new way. Those early con- 
versations were with mainstream customers, not early adopters. 
Still, the conversations yielded a fundamental insight: if Intuit 
could find a way to solve this problem, there could be a large 
mainstream audience on which it could build a significant 
business. 


Design and the Customer Archetype 


The goal of such early contact with customers is not to gain 
definitive answers. Instead, it is to clarify at a basic, coarse level 
that we understand our potential customer and what problems 
they have. With that understanding, we can craft a customer ar- 
chetype, a brief document that seeks to humanize the proposed 
target customer. This archetype is an essential guide for product 
development and ensures that the daily prioritization decisions 
that every product team must make are aligned with the cus- 
tomer to whom the company aims to appeal. 
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There are many techniques for building an accurate customer 
archetype that have been developed over long years of practice in 
the design community. Traditional approaches such as interac- 
tion design or design thinking are enormously helpful. To me, 
it has always seemed ironic that many of these approaches are 
highly experimental and iterative, using techniques such as rapid 
prototyping and in-person customer observations to guide de- 
signers work. Yet because of the way design agencies traditionally 
have been compensated, all this work culminates in a monolithic 
deliverable to the client. All of a sudden, the rapid learning and 
experimentation stops; the assumption is that the designers have 
learned all there is to know. For startups, this is an unworkable 
model. No amount of design can anticipate the many complexi- 
ties of bringing a product to life in the real world. 

In fact, a new breed of designers is developing brand-new 
_ techniques under the banner of Lean User Experience (Lean 
UX). They recognize that the customer archetype is a hypoth- 
esis, not a fact. The customer profile should be considered pro- 
visional until the strategy has shown via validated learning that 
we can serve this type of customer in a sustainable way.’° 


ANALYSIS PARALYSIS 


There are two ever-present dangers when entrepreneurs con- 
duct market research and talk to customers. Followers of the 
just-do-it school of entrepreneurship are impatient to get started 
and dont want to spend time analyzing their strategy. They'd 
rather start building immediately, often after just a few cursory 
customer conversations. Unfortunately, because customers don’t 
really know what they want, it’s easy for these entrepreneurs to 
delude themselves that they are on the right path. 


Other entrepreneurs can fall victim to analysis paralysis, 
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endlessly refining their plans. In this case, talking to custom- 
ers, reading research reports, and whiteboard strategizing are all 
equally unhelpful. The problem with most entrepreneurs’ plans 
is generally not that they don’t follow sound strategic principles 
but that the facts upon which they are based are wrong. Unfor- 
tunately, most of these errors cannot be detected at the white- 
board because they depend on the subtle interactions between 
products and customers. 

If too much analysis is dangerous but none can lead to fail- 
ure, how do entrepreneurs know when to stop analyzing and 
start building? The answer is a concept called the minimum vi- 


able product, the subject of Chapter 6. 
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teen is One of the fastest-growing companies of all time. Its 
name comes from “group coupons,” an ingenious idea that 
has spawned an entire industry of social commerce imitators. 
However, it didn't start out successful. When customers took 
Groupon up on its first deal, a whopping twenty people bought 
two-for-one pizza in a restaurant on the first floor of the com- 
pany's Chicago offices—hardly a world-changing event. 

In fact, Groupon wasn‘ originally meant to be about com- 
~ merce at all. The founder, Andrew Mason, intended his com- 
pany to become a “collective activism platform” called The 
Point. Its goal was to bring people together to solve problems 
they couldn't solve on their own, such as fund-raising for a 
cause or boycotting a certain retailer. The Point’s early re- 
sults were disappointing, however, and at the end of 2008 the 
founders decided to try something new. Although they still 
had grand ambitions, they were determined to keep the new 
product simple. They built a minimum viable product. Does 
this sound like a billion-dollar company to you? Mason tells 
the story: 


We took a WordPress Blog and we skinned it to say 
Groupon and then every day we would do a new post. 
It was totally ghetto. We would sell T-shirts on the first 
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version of Groupon. We'd say in the write-up, “This 
T-shirt will come in the color red, size large. If you want 
a different color or size, e-mail that to us.” We didn't 
have a form to add that stuff. It was just so cobbled 
together. 

It was enough to prove the concept and show that it 
was something that people really liked. The actual cou- 
pon generation that we were doing was all FileMaker. We 
would run a script that would e-mail the coupon PDF 
to people. It got to the point where wed sell 500 sushi 
coupons in a day, and we'd send 500 PDFs to people with 
Apple Mail at the same time. Really until July of the first 
year it was just a scrambling to grab the tiger by the tail. 
It was Eyine to catch up and reasonably piece rv a 
product.' 


Handmade PDFs, a pizza coupon, and a simple blog were 
enough to launch Groupon into record-breaking success; it is 
on pace to become the fastest company in history to achieve 
$1 billion in sales. It is revolutionizing the way local businesses 
find new customers, offering special deals to consumers in more 
than 375 cities worldwide.’ 
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A minimum viable product (MVP) helps entrepreneurs start the 
process of learning as quickly as possible.’ It is not necessarily 
the smallest product imaginable, though; it is simply the fast- 
est way to get through the Build-Measure-Learn feedback loop 
with the minimum amount of effort. 

Contrary to traditional product development, which usu- 
ally involves a long, thoughtful incubation period and strives for 
product perfection, the goal of the MVP is to begin the process 
of learning, not end it. Unlike a prototype or concept test, an 
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MVP is designed not just to answer product design or technical 
questions. Its goal is to test fundamental business hypotheses. 


WHY FIRST PRODUCTS AREN'T MEANT T0 BE PERFECT 


At IMVU, when we were raising money from venture inves- 
tors, we were embarrassed. First of all, our product was still 
buggy and low-quality. Second, although we were proud of our 
business results, they weren't exactly earth-shattering. The good 
news was that we were on a hockey-stick-shaped growth curve. 
The bad news was that the hockey stick went up to only about 
$8,000 per month of revenue. These numbers were so low that 
wed often have investors ask us, “What are the units on these 
charts? Are those numbers in thousands?” We'd have to reply, 
“No, sit, those are in ones.” | 

However, those early results were extremely significant in 
predicting IMVU’s future path. As you'll see in Chapter 7, 
we were able to validate two of our leap-of-faith assumptions: 
IMVU was providing value for customers, and we had a work- 
ing engine of growth. The gross numbers were small because 
we were selling the product to visionary early customers called 
early adopters. Before new products can be sold successfully to 
the mass market, they have to be sold to early adopters. These 
people are a special breed of customer. They accept—in fact 
prefer—an 80 percent solution; you don’t need a perfect solu- 
tion to capture their interest.’ 

Early technology adopters lined up around the block for Ap- 
ple’s original iPhone even though it lacked basic features such 
as copy and paste, 3G Internet speed, and support for corpo- 
rate e-mail. Google's original search engine could answer que- 
ries about specialized topics such as Stanford University and the 
Linux operating system, but it would be years before it could 
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“organize the world’s information.” However, this did not stop 
early adopters from singing its praises. 

Early adopters use their imagination to fill in what a prod- 
uct is missing. They prefer that state of affairs, because what 
they care about above all is being the first to use or adopt a 
new product or technology. In consumer products, it’s often 
the thrill of being the first one on the block to show off a new 
basketball shoe, music player, or cool phone. In enterprise prod- 
ucts, it’s often about gaining a competitive advantage by tak- 
ing a risk with something new that competitors dont have yet. 
Early adopters are suspicious of something that is too polished: 
if it’s ready for everyone to adopt, how much advantage can one 
get by being early? As a result, additional features or polish be- 
yond what early adopters demand is a form of wasted resources 
and time. | 

This is a hard truth for many entrepreneurs to accept. After all, 
the vision entrepreneurs keep in their heads is of a high-quality 
mainstream product that will change the world, not one used by 
a small niche of people who are willing to give it a shot before 
it’s ready. That world-changing product is polished, slick, and 
ready for prime time. It wins awards at trade shows and, most of 
all, is something you can proudly show Mom and Dad. An early, 
buggy, incomplete product feels like an unacceptable compro- 
mise. How many of us were raised with the expectation that we 
would put our best work forward? As one manager put it to me 
recently, “I know for me, the MVP feels a little dangerous—in a 
good way—since I have always been such a perfectionist.” 

Minimum viable products range in complexity from ex- 
tremely simple smoke tests (little more than an advertisement) 
to actual early prototypes complete with problems and miss- 
ing features. Deciding exactly how complex an MVP needs to 
be cannot be done formulaically. It requires judgment. Luckily, 
this judgment is not difficult to develop: most entrepreneurs 
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and product development people dramatically overestimate 
how many features are needed in an MVP. When in doubt, 
simplify. 

For example, consider a service sold with a one-month free 
trial. Before a customer can use the service, he or she has to sign 
up for the trial. One obvious assumption, then, of the business 
model is that customers will sign up for a free trial once they 
have a certain amount of information about the service. A criti- 
cal question to consider is whether customers will in fact sign 
up for the free trial given a certain number of promised features 
(the value hypothesis). 

Somewhere in the business model, probably buried in a sin- 
gle cell in a spreadsheet, it specifies the “percentage of customers 
who see the free trial offer who then sign up.” Maybe in our 
projections we say that this number should be 10 percent. If you 
think about it, this is a leap-of-faith question. It really should 
be represented in giant letters in a bold red font: WE ASSUME 10 
PERCENT OF CUSTOMERS WILL SIGN UP. 

Most entrepreneurs approach a question like this by building 
_ the product and then checking to see how customers react to it. 
I consider this to be exactly backward because it can lead to a 
lot of waste. First, if it turns out that we're building something 
nobody wants, the whole exercise will be an avoidable expense 
of time and money. If customers won't sign up for the free trial, 
they'll never get to experience the amazing features that await 
them. Even if they do sign up, there are many other opportuni- 
ties for waste. For example, how many features do we really need 
to include to appeal to early adopters? Every extra feature is a 
form of waste, and if we delay the test for these extra features, 
it comes with a tremendous potential cost in terms of learning 
and cycle time. 


The lesson of the MVP is that any additional work beyond 
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what was required to start learning is waste, no matter how im- 
portant it might have seemed at the time. 

To demonstrate, I’ll share several MVP techniques from ac- 
tual Lean Startups. In each case, you'll witness entrepreneurs 
avoiding the temptation to overbuild and overpromise. 


THE VIDEO MINIMUM VIABLE PRODUCT 


Drew Houston is the CEO of Dropbox, a Silicon Valley com- 
pany that makes an extremely easy-to-use file-sharing tool. 
Install its application, and a Dropbox folder appears on your 
computer desktop. Anything you drag into that folder is up- 
loaded automatically to the Dropbox service and then instantly 
replicated across all your computers and devices. 

The founding team was made up of engineers, as the product 
demanded significant technical expertise to build. It required, 
for example, integration with a variety of computer platforms 
and operating systems: Windows, Macintosh, iPhone, Android, 
and so on. Each of these implementations happens at a deep 
level of the system and requires specialized know-how to make 
the user experience exceptional. In fact, one of Dropbox’s big- 
gest competitive advantages is that the product works in such a 
seamless way that the competition struggles to emulate it. 

These are not the kind of people one would think of as 
marketing geniuses. In fact, none of them had ever worked in 
_a marketing job. They had prominent venture capital backers 
and could have been expected to apply the standard engineering 
thinking to building the business: build it and they will come. 
But Dropbox did something different. 

In parallel with their product development efforts, the found- 
ers wanted feedback from customers about what really mattered 
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to them. In particular, Dropbox needed to test its leap-of-faith 
question: if we can provide a superior customer experience, 
will people give our product a try? They believed—trightly, as it 
turned out—that file synchronization was a problem that most 
people didn’t know they had. Once you experience the solution, 
you cant imagine how you ever lived without it. 

This is not the kind of entrepreneurial question you can 
ask or expect an answer to in a focus group. Customers often 
don’t know what they want, and they often had a hard time un- 
derstanding Dropbox when the concept was explained. Hous- 
ton learned this the hard way when he tried to raise venture 
capital. In meeting after meeting, investors would explain that 
this “market space” was crowded with existing products, none 
of them had made very much money, and the problem wasn’t 
a very important one. Drew would ask: “Have you personally 
tried those other products?” When they would say yes, he'd ask: 
“Did they work seamlessly for you?” The answer was almost al- 
ways no. Yet in meeting after meeting, the venture capitalists 
could not imagine a world in line with Drew’s vision. Drew, in 
contrast, believed that if the software “just worked like magic,” 
customers would flock to it. 

The challenge was that it was impossible to demonstrate the 
working software in a prototype form. The product required 
that they overcome significant technical hurdles; it also had 
an online service component that required high reliability and 
availability. To avoid the risk of waking up after years of devel- 
opment with a product nobody wanted, Drew did something 
unexpectedly easy: he made a video. - 

The video is banal, a simple three-minute demonstration of 
the technology as it is meant to work, but it was targeted at 
a community of technology early adopters. Drew narrates the 
video personally, and as he’s narrating, the viewer is watching his 
screen. As he describes the kinds of files he'd like to synchronize, 
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the viewer can watch his mouse manipulate his computer. Of 
course, if you're paying attention, you start to notice that the 
files he’s moving around are full of in-jokes and humorous refer- 
ences that were appreciated by this community of early adopt- 
ers. Drew recounted, “It drove hundreds of thousands of people 
to the website. Our beta waiting list went from 5,000 people 
to 75,000 people literally overnight. It totally blew us away.” 
Today, Dropbox is one of Silicon Valley's hottest companies, ru- 
mored to be worth more than $1 billion.” 

In this case, the video was the minimum viable product. The 
MVP validated Drew’s leap-of-faith assumption that customers 
wanted the product he was developing not because they said 
so in a focus group or because of a hopeful analogy to another 
business, but because they actually signed up. 


THE CONCIERGE MINIMUM VIABLE PRODUCT 


Consider another kind of MVP technique: the concierge MVP. 
To understand how this technique works, meet Manuel Rosso, 
the CEO of an Austin, Texas—based startup called Food on the 
Table. Food. on the Table creates weekly meal plans and gro- 
cery lists that are based on food you and your family enjoy, then 
hooks into your local grocery stores to find the best deals on the 
ingredients. 

After you sign up for the site, you walk through a little setup 
in which you identify your main grocery store and check off the 
foods your family likes. Later, you can pick another nearby store 
if you want to compare prices. Next, youre presented with a list 
of items that are based on your preferences and asked: “What 
are you in the mood for this week?” Make your choices, select 
the number of meals you're ready to plan, and choose what you 
care about most in terms of time, money, health, or variety. At 
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this point, the site searches through recipes that match your 
needs, prices out the cost of the meal for you, and lets you print 
out your shopping list.° 

Clearly, this is an elaborate service. Behind the scenes, a 
team of professional chefs devise recipes that take advantage of 
items that are on sale at local grocery stores around the coun- 
try. Those recipes are matched via computer algorithm to each 
family’s unique needs and preferences. Try to visualize the work 
involved: databases of almost every grocery store in the country 
must be maintained, including what’s on sale at each one this 
week. Those groceries have to be matched to appropriate reci- 
pes and then appropriately customized, tagged, and sorted. If a 
recipe calls for broccoli rabe, is that the same ingredient as the 
broccoli on sale at the local market? 

After reading that description, you might be surprised to 
learn that Food on the Table (FotT) began life with a single cus- 
tomer. Instead of supporting thousands of grocery stores around 
the country as it does today, FotT supported just one. How 
did the company choose which store to support? The founders 
didn’t—until they had their first customer. Similarly, they began 
life with no recipes whatsoever—until their first customer was 
ready to begin her meal planning. In fact, the company served 
its first customer without building any software, without sign- 
ing any business development partnerships, and without hiring 
any chefs. 

Manuel, along with VP of product Steve Sanderson, went 
to local supermarkets and moms’ groups in his hometown of 
Austin. Part of their mission was the typical observation of cus- 
tomers that is a part of design thinking and other ideation tech- 
niques. However, Manuel and his team were also on the hunt 
for something else: their first customer. 

As they met potential customers in those settings, they would 
interview them the way any good market researcher would, but 
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.at the end of each interview they would attempt to make a sale. 
They'd describe the benefits of FotT, name a weekly subscrip- 
tion fee, and invite the customer to sign up. Most times they 
were rejected. After all, most people are not early adopters and 
will not sign up for a new service sight unseen. But eventually 
someone did. 

That one early adopter got the concierge treatment. Instead 
of interacting with the FotT product via impersonal software, 
she got a personal visit each week from the CEO of the com- 
pany. He and the VP of product would review what was on sale 
at her preferred grocery store and carefully select recipes on the 
basis of her preferences, going so far as to learn her favorite reci- 
pes for items she regularly cooked for her family. Each week 
they would hand her—in person—a prepared packet containing 
a shopping list and relevant recipes, solicit her feedback, and 
make changes as necessary. Most important, each week they 
would collect a check for $9.95. 

Talk about inefficient! Measured according to traditional cri- 
teria, this is a terrible system, entirely nonscalable and a com- 
plete waste of time. The CEO and VP of product, instead of 
building their business, are engaged in the drudgery of solving 
just one customer’s problem. Instead of marketing themselves to 
millions, they sold themselves to one. Worst of all, their efforts 
didn’t appear to be leading to anything tangible. They had no 
product, no meaningful revenue, no databases of recipes, not 
even a lasting organization. | 

However, viewed through the lens of the Lean Startup, they 
were making monumental progress. Each week they were learning 
more and more about what was required to make their product a 
success. After a few weeks they were ready for another customer. 
Each customer they brought on made it easier to get the next 
one, because FotT could focus on the same grocery store, getting 


to know its products and the kinds of people who shopped there 
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well. Each new customer got the concierge treatment: personal 
in-home visits, the works. But after a few more customers, the 
overhead of serving them one-on-one started to increase. 

Only at the point where the founders were too busy to bring 
on additional customers did Manuel and his team start to invest 
in automation in the form of product development. Each itera- 
tion of their minimum viable product allowed them to save a 
little more time and serve a few more customers: delivering the 
recipes and shopping list via e-mail instead of via an in-home 
visit, starting to parse lists of what was on sale automatically via 
software instead of by hand, even eventually taking credit card 
payments online instead of a handwritten check. 

_ Before long, they had built a substantial service offering, first 
in the Austin area and eventually nationwide. But along the way, 
their product development team was always focused on scaling 
something that was working rather than trying to invent some- 
thing that might work in the future. As a result, their develop- 
ment efforts involved far less waste than is typical for a venture 
of this kind. 

It is important to contrast this with the case of a small busi- 
ness, in which it is routine to see the CEO, founder, president, 
and owner serving customers directly, one at a time. In a con- 
cierge MVP, this personalized service is not the product but a 
learning activity designed to test the leap-of-faith assumptions in 
the company’s growth model. In fact, a common outcome of a 
concierge MVP is to invalidate the company’s proposed growth 
model, making it clear that a different approach is needed. This 
can happen even if the initial MVP is profitable for the com- 
pany. Without a formal growth model, many companies get 
caught in the trap of being satisfied with a small profitable busi- 
ness when a pivot (change in course or strategy) might lead to 
more significant growth. The only way to know is to have tested 
the growth model systematically with real customers. 


Test 108 


PAY NO ATTENTION TO THE EIGHT PEOPLE BEHIND THE CURTAIN 


Meet Max Ventilla and Damon Horowitz, technologists with a 
vision to build a new type of search software designed to answer 
the kinds of questions that befuddle state-of-the-art companies 
such as Google. Google befuddled? Think about it. Google and 
its peers excel at answering factual questions: What is the tall- 
est mountain in the world? Who was the twenty-third presi- 
dent of the United States? But for more subjective questions, 
Google struggles. Ask, “What’s a good place to go out for a 
drink after the ball game in my city?” and the technology flails. 
What's interesting about this class of queries is that they are rela- 
tively easy for a person to answer. Imagine being at a cocktail 
party surrounded by friends. How likely would you be to get 
a high-quality answer to your subjective question? You almost 
certainly would get one. Unlike factual queries, because these 
subjective questions have no single right answer, today’s tech- 
nology struggles to answer them. Such questions depend on the 
person answering them, his or her personal experience, taste, 
and assessment of what you're looking for. 

To solve this problem, Max and Damon created a product 
called Aardvark. With their deep technical knowledge and in- 
dustry experience, it would have been reasonable to expect them 
to dive in and start programming. Instead, they took six months 
to figure out what they should be building. But they didn't 
spend that year at the whiteboard strategizing or engage in a 
lengthy market research project. 

Instead, they built a series of functioning products, each de- 
signed to test a way of solving this problem for their customers. 
Each product was then offered to beta testers, whose behavior 
was used to validate or refute each specific hypothesis (see ex- 
amples in sidebar). 
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The following list of projects are c examples from Aardvarks | 
ideation period.” qe ee | a 


_ Rekkit. Aservice to collect your ratings s from across the web 
‘and give better recommendations to la 


ie Ninjapa. AS way ‘that’ you. could: open accounts in Genone. : 
applications through a single website and mmaliae? your data 
across multiple sites. | 


a The Webb. A ental qamber ‘hae you eld call and calle i 
- to. a person who could do saying o you that you could 
: do online. 


Web Macros. A way to record sequences é steps on websites ny 
so that you could tepeat common actions, even across sites, 
and share “ recipes’ ” for how you ‘accomplished online tasks. 


2 Internet Button. Company. Ay way to package steps taken, | 
- ona website and smart form-fill functionality. People could 
: encode buttons and share buttons? a la social bookmarking, | 


Max and Damon had a vision that computers could be used 
to create a virtual personal assistant to which their customers 
could ask questions. Because the assistant was designed for sub- 
jective questions, the answers required human judgment. Thus, 
the early Aardvark experiments tried many variations on this 
theme, building a series of prototypes for ways customers could 
interact with the virtual assistant and get their questions an- 
swered. All the early prototypes failed to engage the customers. 

As Max describes it, “We self-funded the company and re- 
leased very cheap prototypes to test. What became Aardvark was 
the sixth prototype. Each prototype was a two- to four-week 
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effort. We used humans to replicate the back end as much as 
possible. We invited one hundred to two hundred friends to try 
the prototypes and measured how many of them came back. 
The results were unambiguously negative until Aardvark.” 

Because of the short time line, none of the prototypes involved 
advanced technology. Instead, they were MVPs designed to test a 
more important question: what would be required to get custom- 
ers to engage with the product and tell their friends about it? 

“Once we chose Aardvark,” Ventilla says, “we continued 
to run with humans replicating pieces of the backend for nine 
months. We hired eight people to manage queries, classify con- 
versations, etc. We actually raised our seed and series A rounds 
before the system was automated—the assumption was that the 
lines between humans and artificial intelligence would cross, 
and we at least proved that we were building stuff people would 
respond to. 

“As we refined the product, we would bring in six to twelve 
people weekly to react to mockups, prototypes, or simulations 
that we were working on. It was a mix of existing users and 
people who never saw the product before. We had our engineers 
join for many of these sessions, both so that they could make 
modifications in real time, but also so we could all experience 
the pain of a user not knowing what to do.”® 

The Aardvark product they settled on worked via instant 
messaging (IM). Customers could send Aardvark a question via 
IM, and Aardvark would get them an answer that was drawn 
from the customer’s social network: the system would seek out 
the customer's friends and friends of friends and pose the ques- 
tion to them. Once it got a suitable answer, it would report back 
to the initial customer. 

Of course, a product like that requires a very important al- 
gorithm: given a question about a certain topic, who is the best 


106 THE LEAN STARTUP 


person in the customer's social network to answer that ques- 
tion? For example, a question about restaurants in San Francisco 
shouldn't be routed to someone in Seattle. More challenging still, 
a question about computer programming probably shouldn't be 
routed to an art student. 

Throughout their testing process, Max and Damon encoun- 
tered many difficult technological problems like these. Each 
time, they emphatically refused to solve them at that early stage. 
Instead, they used Wizard of Oz testing to fake it. In a Wizard 
of Oz test, customers believe they are interacting with the ac- 
tual product, but behind the scenes human beings are doing the 
work. Like the concierge MVP, this approach is incredibly inef- 
ficient. Imagine a service that allowed customers to ask ques- 
tions of human researchers—for free—and expect a real-time 
response. Such a service (at scale) would lose money, but it is 
easy to build on a micro scale. At that scale, it allowed Max and 
Damon to answer these all-important questions: If we can solve 
the tough technical problems behind this artificial intelligence 
product, will people use it? Will their use lead to the creation of | 
a product that has real value? 

It was this system that allowed Max and Damon to pivot over 
and over again, rejecting concepts that seemed promising but that 
would not have been viable. When they were ready to start scal- 
ing, they had a ready-made road map of what to build. The result: 
Aardvark was acquired for a reported $50 million—by Google.’ 


THE ROLE OF QUALITY AND DESIGN IN AN MVP 


One of the most vexing aspects of the minimum viable product 
is the challenge it poses to traditional notions of quality. The 
best professionals and craftspersons alike aspire to build quality 


products; it is a point of pride. 
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Modern production processes rely on high quality as a way 
to boost efficiency. They operate using W. Edwards Deming's 
famous dictum that the customer is the most important part 
of the production process. This means that we must focus our 
energies exclusively on producing outcomes that the customer 
perceives as valuable. Allowing sloppy work into our process in- 
evitably leads to excessive variation. Variation in process yields 
products of varying quality in the eyes of the customer that at 
best require rework and at worst lead to a lost customer. Most 
modern business and engineering philosophies focus on pro- 
ducing high-quality experiences for customers as a primary 
principle; it is the foundation of Six Sigma, lean manufacturing, 
design thinking, extreme programming, and the software crafts- 
manship movement. 

These discussions of quality presuppose that the company 
already knows what attributes of the product the customer will 
perceive as worthwhile. In a startup, this is a risky assumption 
to make. Often we are not even sure who the customer is. Thus, 
for startups, I believe in the following quality principle: 


If we do not know who the customer is, we do not know 


what quality is. 


Even a “low-quality” MVP can act in service of building a 
great high-quality product. Yes, MVPs sometimes are perceived 
as low-quality by customers. If so, we should use this as an op- 
portunity to learn what attributes customers care about. This is 
infinitely better than mere speculation or whiteboard strategiz- 
ing, because it provides a solid empirical foundation on which 
to build future products. 

Sometimes, however, customers react quite differently. Many 
famous products were released in a “low-quality” state, and cus- 
tomers loved them. Imagine if Craig Newmark, in the early days 
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of Craigslist, had refused to publish his humble e-mail newslet- 
ter because it lacked sufficient high design. What if the found- 
ers of Groupon had felt “two pizzas for the price of one” was 
beneath them? 

I have had many similar experiences. In the early days of 
IMVU, our avatars were locked in one place, unable to move 
around the screen. The reason? We were building an MVP and 
had not yet tackled the difficult task of creating the technology 
that would allow avatars to walk around the virtual environ- 
ments they inhabit. In the video game industry, the standard 
is that 3D avatars should move fluidly as they walk, avoid ob- 
stacles in their path, and take an intelligent route toward their 
destination. Famous best-selling games such as Electronic Arts’ 
The Sims work on this principle. We didn’t want to ship a 
low-quality version of this feature, so we opted instead to ship 
with stationary avatars. | 

Feedback from the customers was very consistent: they 
wanted the ability to move their avatars around the environ- 
ment. We took this as bad news because it meant we would 
have to spend considerable amounts of time and money on a 
high-quality solution similar to The Sims. But before we com- 
mitted ourselves to that path, we decided to try another MVP. 
We used a simple hack, which felt almost like cheating. We 
changed the product so that customers could click where they 
wanted their avatar to go, and the avatar would teleport there 
instantly. No walking, no obstacle avoidance. The avatar disap- 
peared and then reappeared an instant later in the new place. 
We couldn't even afford fancy teleportation graphics or sound 
effects. We felt lame shipping this feature, but it was all we could 
afford. 

You can imagine our surprise when we started to get positive 
customer feedback. We never asked about the movement feature 
directly (we were too embarrassed). But when asked to name the 
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top things about IMVU they liked best, customers consistently 
listed avatar “teleportation” among the top three (unbelievably, 
they often specifically described it as “more advanced than The 
Sims”). This inexpensive compromise outperformed many fea- 
tures of the product we were most proud of, features that had 
taken much more time and money to produce. 

Customers don’t care how much time something takes to 
build. They care only if it serves their needs. Our customers pre- 
ferred the quick teleportation feature because it allowed them 
to get where they wanted to go as fast as possible. In retrospect, 
this makes sense. Wouldn't we all like to get wherever we're 
going in an instant? No lines, no hours on a plane or sitting on 
the tarmac, no connections, no cabs or subways. Beam me up, 
Scotty. Our expensive “real-world” approach was beaten handily 
by a cool fantasy-world feature that cost much less but that our 
customers preferred. 

So which version of the product is low-quality, again? 

MVPs require the courage to put one’s assumptions to the 
test. If customers react the way we expect, we can take that as 
confirmation that our assumptions are correct. If we release a 
poorly designed product and customers (even early adopters) 
cannot figure out how to use it, that will confirm our need to 
invest in superior design. But we must always ask: what if they 
don’t care about design in the same way we do? 

Thus, the Lean Startup method is not opposed to building 
high-quality products, but only in service of the goal of winning 
over customers. We must be willing to set aside our traditional 
professional standards to start the process of validated learning 
as soon as possible. But once again, this does not mean oper- 
ating in a sloppy or undisciplined way. (This is an important 
caveat. There is a category of quality problems that have the 
net effect of slowing down the Build-Measure-Learn feedback 
loop. Defects make it more difficult to evolve the product. ‘They 
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actually interfere with our ability to learn and so are dangerous 
to tolerate in any production process. We will consider methods 
for figuring out when to make investments in preventing these 
kinds of problems in Part Three.) 

As you consider building your own minimum viable prod- 
uct, let this simple rule suffice: remove any feature, process, or 
effort that does not contribute directly to the learning you seek. 


SPEED BUMPS IN BUILDING AN MVP 


Building an MVP is not without risks, both real and imagined. 
Both can derail a startup effort unless they are understood ahead 
of time. The most common speed bumps are legal issues, fears 
about competitors, branding risks, and the impact on morale. 

For startups that rely on patent protection, there are special 
challenges with releasing an early product. In some jurisdictions, 
the window for filing a patent begins when the product is re- 
leased to the general public, and depending on the way the MVP 
is structured, releasing it may start this clock. Even if your startup 
is not in one of those jurisdictions, you may want international 
patent protection and may wind up having to abide by these 
more stringent requirements. (In my opinion, issues like this are 
one of the many ways in which current patent law inhibits in- 
novation and should be remedied as a matter of public policy.) 

In many industries, patents are used primarily for defensive 
purposes, as a deterrent to hold competitors at bay. In such 
cases, the patent risks of an MVP are minor compared with the 
learning benefits. However, in industries in which a new sci- 
entific breakthrough is at the heart of a company’s competitive 
advantage, these risks need to be balanced more carefully. In 
all cases, entrepreneurs should seek legal counsel to ensure that 
they understand the risks fully. 
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Legal risks may be daunting, but you may be surprised to 
learn that the most common objection I have heard over the 
years to building an MVP is fear of competitors—especially large 
established companies—stealing a startup’s ideas. If only it were 
so easy to have a good idea stolen! Part of the special challenge 
of being a startup is the near impossibility of having your idea, 
company, or product be noticed by anyone, let alone a competi- 
tor. In fact, I have often given entrepreneurs fearful of this issue 
the following assignment: take one of your ideas (one of your 
lesser insights, perhaps), find the name of the relevant product 
manager at an established company who has responsibility for 
that area, and try to get that company to steal your idea. Call 
them up, write them a memo, send them a press release—go 
ahead, try it. The truth is that most managers in most compa- 
nies are already overwhelmed with good ideas. Their challenge 
lies in prioritization and execution, and it is those challenges 
that give a startup hope of surviving.'® 

If a competitor can outexecute a startup once the idea is 
known, the startup is doomed anyway. The reason to build a 
new team to pursue an idea is that you believe you can acceler- 
ate through the Build-Measure-Learn feedback loop faster than 
anyone else can. If that’s true, it makes no difference what the 
competition knows. If it’s not true, a startup has much bigger 
problems, and secrecy won't fix them. Sooner or later, a suc- 
cessful startup will face competition from fast followers. A head 
start is rarely large enough to matter, and time spent in stealth 
mode—away from customers—is unlikely to provide a head 
start. The only way to win is to learn faster than anyone else. 

Many startups plan to invest in building a great brand, and 
an MVP can seem like a dangerous branding risk. Similarly, en- 
trepreneurs in existing organizations often are constrained by 
the fear of damaging the parent company’s established brand. In 
either of these cases, there is an easy solution: launch the MVP 
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under a different brand name. In addition, a long-term repu- 
tation is only at risk when companies engage in vocal launch 
activities such as PR and building hype. When a product fails 
to live up to those pronouncements, real long-term damage can 
happen to a corporate brand. But startups have the advantage of 
being obscure, having a pathetically small number of customers, 
and not having much exposure. Rather than lamenting them, 
use these advantages to experiment under the radar and then 
do a public marketing launch once the product has proved itself 
with real customers." 

Finally, it helps to prepare for the fact that MVPs often re- 
sult in bad news. Unlike traditional concept tests or prototypes, 
they are designed to speak to the full range of business ques- 
tions, not just design or technical ones, and they often provide 
a needed dose of reality. In fact, piercing the reality distortion 
field is quite uncomfortable. Visionaries are especially afraid of 
a false negative: that customers will reject a flawed MVP that is 
too small or too limited. It is precisely this attitude that one sees 
when companies launch fully formed products without prior 
testing. They simply couldn't bear to test them in anything less 
than their full splendor. Yet there is wisdom in the visionary’s 
fear. Teams steeped in traditional product development meth- 
ods are trained to make go/kill decisions on a regular basis. That 
is the essence of the waterfall or stage-gate development model. 
If an MVP fails, teams are liable to give up hope and abandon 
the project altogether. But this is a solvable problem. 


FROM THE MVP TO INNOVATION ACCOUNTING 


The solution to this dilemma is a commitment to iteration. You 
have to commit to a locked-in agreement—ahead of time—that 
no matter what comes of testing the MVP, you will not give 
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up hope. Successful entrepreneurs do not give up at the first 
sign of trouble, nor do they persevere the plane right into the 
ground. Instead, they possess a unique combination of persever- 
ance and flexibility. The MVP is just the first step on a journey 
of learning. Down that road—after many iterations—you may 
learn that some element of your product or strategy is flawed 
and decide it is time to make a change, which I call a pivot, to a 
different method for achieving your vision. 

Startups are especially at risk when outside stakeholders 
and investors (especially corporate CFOs for internal projects) 
have a crisis of confidence. When the project was authorized or 
the investment made, the entrepreneur promised that the new 
product would be world-changing. Customers were supposed to 
flock to it in record numbers. Why are so few actually doing so? 

In traditional management, a manager who promises to de- 
liver something and fails to do so is in trouble. There are only 
two possible explanations: a failure of execution or a failure to 
plan appropriately. Both are equally inexcusable. Entrepreneur- 
ial managers face a difficult problem: because the plans and 
projections we make are full of uncertainty, how can we claim 
success when we inevitably fail to deliver what we promised? Put 
another way, how does the CFO or VC know that we're failing 
because we learned something critical and not because we were 
goofing off or misguided? 

The solution to this problem resides at the heart of the 
Lean Startup model. We all need a disciplined, systematic ap- 
proach to figuring out if we're making progress and discovering 
if we're actually achieving validated learning. I call this system 
innovation accounting, an alternative to traditional accounting 


designed specifically for startups. It is the subject of Chapter 7. 


{ 
MEASURE 


{the beginning, a startup is little more than a model on a piece of 

paper. The financials in the business plan include projections 
of how many customers the company expects to attract, how 
much it will spend, and how much revenue and profit that will 
lead to. It’s an ideal that’s usually far from where the startup is in 
its early days. 

A startup’s job is to (1) rigorously measure where it is right 
now, confronting the hard truths that assessment reveals, and 
then (2) devise experiments to learn how to move the real num- 
bers closer to the ideal reflected in the business plan. 

Most products—even the ones that fail—do not have zero 
traction. Most products have some customers, some growth, 
and some positive results. One of the most dangerous out- 
comes for a startup is to bumble along in the land of the living 
dead. Employees and entrepreneurs tend to be optimistic by 
nature. We want to keep believing in our ideas even when the 
writing is on the wall. This is why the myth of perseverance is 
so dangerous. We all know stories of epic entrepreneurs who 
managed to pull out a victory when things seemed incredibly 
bleak. Unfortunately, we don’t hear stories about the countless 
nameless others who persevered too long, leading their com- 
panies to failure. | 
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WHY SOMETHING AS SEEMINGLY DULL AS ACCOUNTING 
WILL CHANGE YOUR LIFE 


People are accustomed to thinking of accounting as dry and 
boring, a necessary evil used primarily to prepare financial 
reports and survive audits, but that is because accounting is 
something that has become taken for granted. Historically, 
under the leadership of people such as Alfred Sloan at General 
Motors, accounting became an essential part of the method of 
exerting centralized control over far-flung divisions. Account- 
ing allowed GM to set clear milestones for each of its divisions 
and then hold each manager accountable for his or her divi- 
sion’s success in reaching those goals. All modern corporations 
use some variation of that approach. Accounting is the key to 
their success. 

Unfortunately, standard accounting is not helpful in evaluat- 
ing entrepreneurs. Startups are too unpredictable for forecasts 
and milestones to be accurate. 

I recently met with a phenomenal startup team. They are 
well financed, have significant customer traction, and are grow- 
ing rapidly. Their product is a leader in an emerging category 
of enterprise software that uses consumer marketing tech- 
niques to sell into large companies. For example, they rely on 
employee-to-employee viral adoption rather than a traditional 
sales process, which might target the chief information officer or 
the head of information technology (IT). As a result, they have 
the opportunity to use cutting-edge experimental techniques 
as they constantly revise their product. During the meeting, I 
asked the team a simple question that I make a habit of ask- 
ing startups whenever we meet: are you making your product 
better? They always say yes. Then I ask: how do you know? I 
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invariably get this answer: well, we are in engineering and we 
made a number of changes last month, and our customers seem 
to like them, and our overall numbers are higher this month. 
We must be on the right track. 

This is the kind. of storytelling that takes place at most 
startup board meetings. Most milestones are built the same way: 
hit a certain product milestone, maybe talk to a few customers, 
and see if the numbers go up. Unfortunately, this is not a good 
indicator of whether a startup is making progress. How do we 
know that the changes we've made are related to the results we're 
seeing? More important, how do we know that we are drawing 
the right lessons from those changes? 

To answer these kinds of questions, startups have a strong 
need for a new kind of accounting geared specifically to disrup- 
tive innovation. That's what innovation accounting is. 


An Accountability Framework That Works Across Industries 


Innovation accounting enables startups to prove objectively that 
they are learning how to grow a sustainable business. Innovation 
accounting begins by turning the leap-of-faith assumptions dis- 
cussed in Chapter 5 into a quantitative financial model. Every 
business plan has some kind of model associated with it, even 
if it’s written on the back of a napkin. That model provides as- 
sumptions about what the business will look like at a successful 
point in the future. 

For example, the business plan for an established manufac- 
turing company would show it growing in proportion to its sales 
volume. As the profits from the sales of goods are reinvested in 
marketing and promotions, the company gains new custom- 
ers. The rate of growth depends primarily on three things: the 
profitability of each customer, the cost of acquiring new cus- 
tomers, and the repeat purchase rate of existing customers. The 
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higher these values are, the faster the company will grow and the 
more profitable it will be. These are the drivers of the company's 
growth model. | 

By contrast, a marketplace company that matches buyers and 
sellers such as eBay will have a different growth model. Its suc- 
cess depends primarily on the network effects that make it the 
premier destination for both buyers and sellers to transact busi- 
ness. Sellers want the marketplace with the highest number of 
potential customers. Buyers want the marketplace with the most 
competition among sellers, which leads to the greatest availabil- 
ity of products and the lowest prices. (In economics, this some- 
times is called supply-side increasing returns and demand-side 
increasing returns.) For this kind of startup, the important thing 
to measure is that the network effects are working, as evidenced 
by the high retention rate of new buyers and sellers. If people 
stick with the product with very little attrition, the marketplace 
will grow no matter how the company acquires new customers. 
The growth curve will look like a compounding interest table, 
with the rate of growth depending on the “interest rate” of new 
customers coming to the product. 

Though these two businesses have very different drivers of 
growth, we can still use a common framework to hold their 
leaders accountable. This framework supports accountability 
even when the model changes. 


HOW INNOVATION ACCOUNTING WORKS—THREE LEARNING MILESTONES 


Innovation accounting works in three steps: first, use a mini- 
mum viable product to establish real data on where the com- 
pany is right now. Without a clear-eyed picture of your current 
status—no matter how far from the goal you may be—you can- 
not begin to track your progress. 
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Second, startups must attempt to tune the engine from the 
baseline toward the ideal. This may take many attempts. After 
the startup has made all the micro changes and product opti- 
mizations it can to move its baseline toward the ideal, the com- 
pany reaches a decision point. That is the third step: pivot or 
persevere. 

If the company is making good progress toward the ideal, 
that means it’s learning appropriately and using that learning 
effectively, in which case it makes sense to continue. If not, the 
management team eventually must conclude that its current 
product strategy is flawed and needs a serious change. When a 
company pivots, it starts the process all over again, reestablish- 
ing a new baseline and then tuning the engine from there. The 
sign of a successful pivot is that these engine-tuning activities 
are more productive after the pivot than before. 


Establish the Baseline 


For example, a startup might create a complete prototype of its 
product and offer to sell it to real customers through its main 
marketing channel. This single MVP would test most of the 
startups assumptions and establish baseline metrics for each as- 
sumption simultaneously. Alternatively, a startup might prefer 
to build separate MVPs that are aimed at getting feedback on 
one assumption at a time. Before building the prototype, the 
company might perform a smoke test with its marketing materi- 
als. This is an old direct marketing technique in which custom- 
ers are given the opportunity to preorder a product that has not 
_ yet been built. A smoke test measures only one thing: whether 
customers are interested in trying a product. By itself, this is 
insufficient to validate an entire growth model. Nonetheless, it 
can be very useful to get feedback on this assumption before 
committing more money and other resources to the product. 


Measure 119 


These MVPs provide the first example of a learning milestone. 
An MVP allows a startup to fill in real baseline data in its growth 
model—conversion rates, sign-up and trial rates, customer life- 
time value, and so on—and this is valuable as the foundation for 
learning about customers and their reactions to a product even 
if that foundation begins with extremely bad news. 

When one is choosing among the many assumptions in a 
business plan, it makes sense to test the riskiest assumptions 
first. If you can’t find a way to mitigate these risks toward the 
ideal that is required for a sustainable business, there is no point 
in testing the others. For example, a media business that is sell- 
ing advertising has two basic assumptions that take the form of 
questions: Can it capture the attention of a defined customer 
segment on an ongoing basis? and can it sell that attention to 
advertisers? In a business in which the advertising rates for a 
particular customer segment are well known, the far riskier as- 
sumption is the ability to capture attention. Therefore, the first 
experiments should involve content production rather than ad- 
vertising sales. Perhaps the company will produce a pilot episode 
or issue to see how customers engage. 


Tuning the Engine 


Once the baseline has been established, the startup can work 
toward the second learning milestone: tuning the engine. Every 
product development, marketing, or other initiative that a 
startup undertakes should be targeted at improving one of the 
drivers of its growth model. For example, a company might 
spend time improving the design of its product to make it easier 
for new customers to use. This presupposes that the activation 
rate of new customers is a driver of growth and that its baseline 
is lower than the company would like. To demonstrate validated 
learning, the design changes must improve the activation rate of 
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new customers. If they do not, the new design should be judged 
a failure. This is an important rule: a good design is one that 
changes customer behavior for the better. 

Compare two startups. The first company sets out with a 
clear baseline metric, a hypothesis about what will improve that 
metric, and a set of experiments designed to test that hypothesis. 
The second team sits around debating what would improve the 
product, implements several of those changes at once, and cel- 
ebrates if there is any positive increase in any of the numbers. 
Which startup is more likely to be doing effective work and 
achieving lasting results? 


Pivot or Persevere 


Over time, a team that is learning its way toward a sustainable 
business will see the numbers in its model rise from the horrible 
baseline established by the MVP and converge to something like 
the ideal one established in the business plan. A startup that 
fails to do so will see that ideal recede ever farther into the dis- 
tance. When this is done right, even the most powerful reality 
distortion field won't be able to cover up this simple fact: if we're 
not moving the drivers of our business model, we're not making | 
progress. That becomes a sure sign that it’s time to pivot. 


INNOVATION ACCOUNTING AT IMVU 


Here's what innovation accounting looked like for us in the 
early days of IMVU. Our minimum viable product had many 
defects and, when we first released it, extremely low sales. We 
naturally assumed that the lack of sales was related to the low 
quality of the product, so week after week we worked on im- 
proving the quality of the product, trusting that our efforts were 
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worthwhile. At the end of each month, we would have a board 
meeting at which we would present the results. The night before 
the board meeting, wed run our standard analytics, measuring 
conversion rates, customer counts, and revenue to show what a 
good job we had done. For several meetings in a row, this caused 
a last-minute panic because the quality improvements were not 
yielding any change in customer behavior. This led to some 
frustrating board meetings at which we could show great prod- 
uct “progress” but not much in the way of business results. After 
a while, rather than leave it to the last minute, we began to track 
our metrics more frequently, tightening the feedback loop with 
product development. This was even more depressing. Week in, 
week out, our product changes were having no effect. 


Improving a Product on Five Dollars a Day 


We tracked the “funnel metrics” behaviors that were critical 
to our engine of growth: customer registration, the download 
of our application, trial, repeat usage, and purchase. To have 
enough data to learn, we needed just enough customers using 
our product to get real numbers for each behavior. We allocated 
a budget of five dollars per day: enough to buy clicks on the 
then-new Google AdWords system. In those days, the minimum 
you could bid for a click was 5 cents, but there was no overall 
minimum to your spending. Thus, we could afford to open an 
account and get started even though we had very little money.’ 

Five dollars bought us a hundred clicks—every day. From 
a marketing point of view this was not very significant, but for 
learning it was priceless. Every single day we were able to mea- 
sure our product's performance with a brand new set of custom- 
ers. Also, each time we revised the product, we got a brand new 
report card on how we were doing the very next day. 

For example, one day we would debut a new marketing 
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message aimed at first-time customers. The next day we might 
change the way new customers were initiated into the product. 
Other days, we would add new features, fix bugs, roll out a new 
visual design, or try a new layout for our website. Every time, 
we told ourselves we were making the product better, but that 
subjective confidence was put to the acid test of real numbers. 

Day in and day out we were performing random trials. Each 
day was a new experiment. Each day's customers were indepen- 
dent of those of the day before. Most important, even though 
our gross numbers were growing, it became clear that our funnel 
metrics were not changing. 

Here is a graph from one of IMVU’s early board meetings: 
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This graph represents approximately seven months of work. 
Over that period, we were making constant improvements to 
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the IMVU product, releasing new features on a daily basis. We 
were conducting a lot of in-person customer interviews, and our 
product development team was working extremely hard. 


Cohort Analysis 


To read the graph, you need to understand something called 
cohort analysis. This is one of the most important tools of startup 
analytics. Although it sounds complex, it is based on a simple 
premise. Instead of looking at cumulative totals or gross num- 
bers such as total revenue and total number of customers, one 
looks at the performance of each group of customers that comes 
into contact with the product independently. Each group is 
called a cohort. The graph shows the conversion rates to IMVU 
of new customers who joined in each indicated month. Each 
conversion rate shows the percentage of customer who regis- 
tered in that month who subsequently went on to take the indi- 
cated action. Thus, among all the customers who joined IMVU 
in February 2005, about 60 percent of them logged in to our 
product at least one time. 

Managers with an enterprise sales background will recognize 
this funnel analysis as the traditional sales funnel that is used to 
manage prospects on their way to becoming customers. Lean 
Startups use it in product development, too. This technique is 
useful in many types of business, because every company de- 
pends for its survival on sequences of customer behavior called 
flows. Customer flows govern the interaction of customers with 
a company’s products. They allow us to understand a business 
quantitatively and have much more predictive power than do 
traditional gross metrics. 

If you look closely, you'll see that the graph shows some clear 
trends. Some product improvements are helping—a little. ‘The 
percentage of new customers who go on to use the product at 
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least five times has grown from less than 5 percent to almost 20 
percent. Yet despite this fourfold increase, the percentage of new 
customers who pay money for IMVU is stuck at around 1 per- 
cent. Think about that for a moment. After months and months 
of work, thousands of individual improvements, focus groups, 
design sessions, and usability tests, the percentage of new cus- 
tomers who subsequently pay money is exactly the same as it 
was at the onset even though many more customers are getting 
a chance to try the product. 

Thanks to the power of cohort analysis, we could not blame 
this failure on the legacy of previous customers who were resis- 
tant to change, external market conditions, or any other excuse. 
Each cohort represented an independent report card, and try as 
we might, we were getting straight C’s. This helped us realize we 
had a problem. 

I was in charge of the product development team, small 
though it was in those days, and shared with my cofounders 
the sense that the problem had to be with my team’s efforts. 
I worked harder, tried to focus on higher- and higher-quality 
features, and lost a lot of sleep. Our frustration grew. When I 
could think of nothing else to do, I was finally ready to turn to 
the last resort: talking to customers. Armed with our failure to 
make progress tuning our engine of growth, I was ready to ask 
the right questions. 

Before this failure, in the company’s earliest days, it was easy 
to talk to potential customers and come away convinced we 
were on the right track. In fact, when we would invite custom- 
ers into the office for in-person interviews and usability tests, it 
was easy to dismiss negative feedback. If they didn't want to use 
the product, I assumed they were not in our target market. “Fire 
that customer,” I'd say to the person responsible for recruiting 
for our tests. “Find me someone in our target demographic.” If 
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the next customer was more positive, I would take it as confir- 
mation that I was right in my targeting. If not, I'd fire another 
customer and try again. 

By contrast, once I had data in hand, my interactions with 
customers changed. Suddenly I had urgent questions that needed 
answering: Why aren't customers responding to our product “im- 
provements”? Why isn’t our hard work paying off? For example, 
we kept making it easier and easier for customers to use IMVU 
with their existing friends. Unfortunately, customers didn’t want 
to engage in that behavior. Making it easier to use was totally 
beside the point. Once we knew what to look for, genuine un- 
derstanding came much faster. As was described in Chapter 3, 
this eventually led to a critically important pivot: away from an 
IM add-on used with existing friends and toward a stand-alone 
network one can use to make new friends. Suddenly, our worries 
about productivity vanished. Once our efforts were aligned with 
what customers really wanted, our experiments were much more 
likely to change their behavior for the better. 

This pattern would repeat time and again, from the days 
when we were making less than a thousand dollars in revenue 
per month all the way up to the time we were making millions. 
In fact, this is the sign of a successful pivot: the new experiments 
you run are overall more productive than the experiments you 
were running before. 

This is the pattern: poor quantitative results force us to de- 
clare failure and create the motivation, context, and space for 
more qualitative research. These investigations produce new 
ideas—new hypotheses—to be tested, leading to a possible 
pivot. Each pivot unlocks new opportunities for further experi- 
mentation, and the cycle repeats. Each time we repeat this sim- 
ple rhythm: establish the baseline, tune the engine, and make a 
decision to pivot or persevere. 
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OPTIMIZATION VERSUS LEARNING 


Engineers, designers, and marketers are all skilled at optimi- 
zation. For example, direct marketers are experienced at split — 
testing value propositions by sending a different offer to two 
similar groups of customers so that they can measure differences 
in the response rates of the two groups. Engineers, of course, are 
skilled at improving a product’s performance, just as designers 
are talented at making products easier to use. All these activities 
in a well-run traditional organization offer incremental benefit 
for incremental effort. As long as we are executing the plan well, 
hard work yields results. 

However, these tools for product improvement do not work 
the same way for startups. If you are building the wrong thing, 
optimizing the product or its marketing will not yield signifi- 
cant results. A startup has to measure progress against a high 
bar: evidence that a sustainable business can be built around its 
products or services. That’s a standard that can be assessed only 
if a startup has made clear, tangible predictions ahead of time. 

In the absence of those predictions, product and strategy de- 
cisions are far more difficult and time-consuming. I often see 
this in my consulting practice. I’ve been called in many times to 
help a startup that feels that its engineering team “isn’t working 
hard enough.” When I meet with those teams, there are always 
improvements to be made and I recommend them, but invari- 
ably the real problem is not a lack of development talent, energy, 
or effort. Cycle after cycle, the team is working hard, but the 
business is not seeing results. Managers trained in a traditional 
model draw the logical conclusion: our team is not working 
hard, not working effectively, or not working efficiently. 

Thus the downward cycle begins: the product develop- 
ment team valiantly tries to build a product according to the 
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specifications it is receiving from the creative or business lead- 
ership. When good results are not forthcoming, business lead- 
ers assume that any discrepancy between what was planned and 
what was built is the cause and try to specify the next itera- 
tion in greater detail. As the specifications get more detailed, the 
planning process slows down, batch size increases, and feedback 
is delayed. If a board of directors or CFO is involved as a stake- 
holder, it doesn’t take long for personnel changes to follow. 

__A few years ago, a team that sells products to large media 
companies invited me to help them as a consultant because 
they were concerned that their engineers were not working hard 
enough. However, the fault was not in the engineers; it was in 
the process the whole company was using to make decisions. 
They had customers but did not know them very well. They 
were deluged with feature requests from customers, the internal 
sales team, and the business leadership. Every new insight be- 
came an emergency that had to be addressed immediately. As a 
result, long-term projects were hampered by constant interrup- 
tions. Even worse, the team had no clear sense of whether any 
of the changes they were making mattered to customers. Despite 
the constant tuning and tweaking, the business results were con- 
sistently mediocre. 

Learning milestones prevent this negative spiral by empha- 
sizing a more likely possibility: the company is executing—with 
discipline!—a plan that does not make sense. The innovation 
accounting framework makes it clear when the company is stuck 
and needs to change direction. 

In the example above, early in the company’s life, the product 
development team was incredibly productive because the com- 
pany’s founders had identified a large unmet need in the tar- 
get market. The initial product, while flawed, was popular with 
early adopters. Adding the major features that customers asked 
for seemed to work wonders, as the early adopters spread the 
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word about the innovation far and wide. But unasked and un- 
answered were other lurking questions: Did the company have a 
working engine of growth? Was this early success related to the 
daily work of the product development team? In most cases, the _ 
answer was no; success was driven by decisions the team had 
made in the past. None of its current initiatives were having 
any impact. But this was obscured because the company’s gross 
metrics were all “up and to the right.” | 

As well see in a moment, this is a common danger. Com- 
panies of any size that have a working engine of growth can 
come to rely on the wrong kind of metrics to guide their ac- 
tions. This is what tempts managers to resort to the usual bag 
of success theater tricks: last-minute ad buys, channel stuffing, 
and whiz-bang demos, in a desperate attempt to make the gross 
numbers look better. Energy invested in success theater is energy 
that could have been used to help build a sustainable business. I 
call the traditional numbers used to judge startups “vanity met- 
rics,” and innovation accounting requires us to avoid the temp- 
tation to use them. 


VANITY METRICS: A WORD OF CAUTION 


To see the danger of vanity metrics clearly, let’s return once 
more to the early days of IMVU. Take a look at the following 
graph, which is from the same era in IMVU’s history as that 
shown earlier in this chapter. It covers the same time period as 
the cohort-style graph on page 122; in fact, it is from the same 
board presentation. 

This graph shows the traditional gross metrics for IMVU so 
far: total registered users and total paying customers (the gross 
revenue graph looks almost the same). From this viewpoint, 
things look much more exciting. That’s why I call these vanity 
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metrics: they give the rosiest possible picture. You'll see a tra- 
ditional hockey stick graph (the ideal in a rapid-growth com- 
pany). As long as you focus on the top-line numbers (signing 
up more customers, an increase in overall revenue), you ll be 
forgiven for thinking this product development team is making 
great progress. The company’s growth engine is working. Each 
month it is able to acquire customers and has a positive return 
on investment. The excess revenue from those customers is re- 
invested the next month in acquiring more. That’s where the 
growth is coming from. 

But think back to the same data presented in a cohort style. 
IMVU is adding new customers, but it is not improving the 
yield on each new group. The engine is turning, but the efforts 
to tune the engine are not bearing much fruit. From the tradi- 
tional graph alone, you cannot tell whether IMVU is on pace 
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to build a sustainable business; you certainly can’t tell anything 
about the efficacy of the entrepreneurial team behind it. 

Innovation accounting will not work if a startup is being 
misled by these kinds of vanity metrics: gross number of cus- 
tomers and so on. The alternative is the kind of metrics we use 
to judge our business and our learning aresone what I call 
actionable metrics. 


ACTIONABLE METRICS VERSUS VANITY METRICS 


To get a better sense of the importance of good metrics, let’s 
look at a company called Grockit. Its founder, Farbood Nivi, 
spent a decade working as a teacher at two large for-profit edu- 
cation companies, Princeton Review and Kaplan, helping stu- 
dents prepare for standardized tests such as the GMAT, LSAT, 
and SAT. His engaging classroom style won accolades from his 
students and promotions from his superiors; he was honored 
with Princeton Review's National Teacher of the Year award. 
But Farb was frustrated with the traditional teaching methods 
used by those companies. Teaching six to nine hours per day 
to thousands of students, he had many opportunities to experi- 
ment with new approaches.? 

Over time, Farb concluded that the traditional lecture model 
of education, with its one-to-many instructional approach, was 
inadequate for his students. He set out to develop a superior ap- 
proach, using a combination of teacher-led lectures, individual 
homework, and group study. In particular, Farb was fascinated 
by how effective the student-to-student peer-driven learning 
method was for his students. When students could help each 
other, they benefited in two ways. First, they could get custom- 
ized instruction from a peer who was much less intimidating 
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than a teacher. Second, they could reinforce their learning by 
teaching it to others. Over time, Farb’s classes became increas- 
ingly social—and successful. 

As this unfolded, Farb felt more and more that his physi- 
cal presence in the classroom was less important. He made an 
important connection: “I have this social learning model in my 
classroom. There’s all this social stuff going on on the web.” His 
idea was to bring social peer-to-peer learning to people who 
could not afford an expensive class from Kaplan or Princeton 
Review or an even more expensive private tutor. From this in- 
sight Grockit was born. : 

Farb explains, “Whether youre studying for the SAT or 
youre studying for algebra, you study in one of three ways. You 
spend some time with experts, you spend some time on your 
own, and you spend some time with your peers. Grockit offers 
these three same formats of studying. What we do is we apply 
technology and algorithms to optimize those three forms.” 

Farb is the classic entrepreneurial visionary. He recounts his 
original insight this way: “Let’s forget educational design up 
until now, let’s forget what's possible and just redesign learn- 
ing with today’s students and today’s technology in mind. There 
were plenty of multi-billion-dollar organizations in the educa- 
tion space, and I don’t think they were innovating in the way 
that we needed them to and I didn’t think we needed them any- 
more. To me, it’s really all about the students and I didn’t feel 
like the students were being served as well as they could.” 

Today Grockit offers many different educational products, 
but in the beginning Farb followed a lean approach. Grockit 
built a minimum viable product, which was simply Farb teach- 
ing test prep via the popular online web conferencing tool 
WebEx. He built no custom software, no new technology. 
He simply attempted to bring his new teaching approach to 
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students via the Internet. News about a new kind of private tu- 
toring spread quickly, and within a few months Farb was mak- 
ing a decent living teaching online, with monthly revenues of 
$10,000 to $15,000. But like many entrepreneurs with ambi- 
tion, Farb didn’t build his MVP just to make a living. He had a 
vision of a more collaborative, more effective kind of teaching 
for students everywhere. With his initial traction, he was able 
to raise money from some of the most prestigious investors in 
Silicon Valley. | 

When I first met Farb, his company was already on the 
fast track to success. They had raised venture capital from 
well-regarded investors, had built an awesome team, and were 
fresh off an impressive debut at one of Silicon Valley's famous 
startup Competitions. 

They were extremely process-oriented and disciplined. Their 
product development followed a rigorous version of the agile 
development methodology known as Extreme Programming 
(described below), thanks to their partnership with a San Fran- 
cisco—based company called Pivotal Labs. Their early product 
was hailed by the press as a breakthrough. 

There was only one problem: they were not seeing sufficient 
growth in the use of the product by customers. Grockit is an 
excellent case study because its problems were not a matter of 
failure of execution or discipline. 

Following standard agile practice, Grockit’s work proceeded 
in a series of sprints, or one-month iteration cycles. For each 
sprint, Farb would prioritize the work to be done that month by 
writing a series of user stories, a technique taken from agile devel- 
opment. Instead of writing a specification for a new feature that 
described it in technical terms, Farb would write a story that 
described the feature from the point of view of the customer. 
That story helped keep the engineers focused on the customer's 
perspective throughout the development process. 
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Each feature was expressed in plain language in terms every- 
one could understand whether they had a technical background 
or not. Again following standard agile practice, Farb was free to 
reprioritize these stories at any time. As he learned more about 
what customers wanted, he could move things around in the 
product backlog, the queue of stories yet to be built. The only 
limit on this ability to change directions was that he could not 
interrupt any task that was in progress. Fortunately, the stories 
were written in such a way that the batch size of work (which 
Pll discuss in more detail in Chapter 9) was only a day or two. 

This system is called agile development for a good reason: 
teams that employ it are able to change direction quickly, stay 
light on their feet, and be highly responsive to changes in the 
business requirements of the product owner (the manager of the 
process—in this case Farb—who is responsible for prioritizing 
the stories). 

How did the team feel at the end of each sprint? They consis- 
tently delivered new product features. They would collect feed- 
back from customers in the form of anecdotes and interviews 
that indicated that at least some customers liked the new features. 
There was always a certain amount of data that showed improve- 
ment: perhaps the total number of customers was increasing, the 
total number of questions answered by students was going up, or 
the number of returning customers was increasing. 

However, I sensed that Farb and his team were left with lin- 
gering doubts about the company’s overall progress. Was the in- 
crease in their numbers actually caused by their development 
efforts? Or could it be due to other factors, such as mentions of 
Grockit in the press? When I met the team, I asked them this 
simple question: How do you know that the prioritization deci- 
sions that Farb is making actually make sense? 

Their answer: “That's not our department. Farb makes the 
decisions; we execute them.” 
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At that time Grockit was focused on just one customer seg- 
ment: prospective business school students who were studying 
for the GMAT. The product allowed students to engage in on- 
line study sessions with fellow students who were studying for 
the same exam. The product was working: the students who 
completed their studying via Grockit achieved significantly 
higher scores than they had before. But the Grockit team was 
struggling with the age-old startup problems: How do we know 
which features to prioritize? How can we get more customers 
to sign up and pay? How can we get out the word about our 
product? 

I put this question to Farb: “How confident are you that you 
are making the right decisions in terms of establishing priori-_ 
ties?” Like most startup founders, he was looking at the available 
data and making the best educated guesses he could. But this 
left a lot of room for ambiguity and doubt. 

Farb believed in his vision thoroughly and completely, yet 
he was starting to question whether his company was on pace 
to realize that vision. The product improved every day, but Farb 
wanted to make sure those improvements mattered to custom- 
ers. I believe he deserves a lot of credit for realizing this. Unlike 
many visionaries, who cling to their original vision no matter 
what, Farb was willing to put his vision to the test. 

Farb worked hard to sustain his team’s belief that Grockit 
was destined for success. He was worried that morale would suf- 
fer if anyone thought that the person steering the ship was un- 
certain about which direction to go. Farb himself wasn’t sure if 
his team would embrace a true learning culture. After all, this 
was part of the grand bargain of agile development: engineers 
agree to adapt the product to the business’s constantly chang- 
ing requirements but are not responsible for the quality of those 
business decisions. 

Agile is an efficient system of development from the point of 
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view of the developers. It allows them to stay focused on creat- 
ing features and technical designs. An attempt to introduce the 
need to learn into that process could undermine productivity. 

(Lean manufacturing faced similar problems when it was in- 
troduced in factories. Managers were used to focusing on the 
utilization rate of each machine. Factories were designed to keep 
machines running at full capacity as much of the time as pos- 
sible. Viewed from the perspective of the machine, that is ef- 
ficient, but from the point of view of the productivity of the 
entire factory, it is wildly inefficient at times. As they say in sys- 
tems theory, that which optimizes one part of the system neces- 
sarily undermines the system as a whole.) 

What Farb and his team didn’t realize was that Grockit’s 
progress was being measured by vanity metrics: the total num- 
ber of customers and the total number of questions answered. 
That was what was causing his team to spin its wheels; those 
metrics gave the team the sensation of forward motion even 
though the company was making little progress. What's inter- 
esting is how closely Farb’s method followed superficial aspects 
of the Lean Startup learning milestones: they shipped an early 
product and established some baseline metrics, They had rela- 
tively short iterations, each of which was judged by its ability to 
improve customer metrics. 

However, because Grockit was using the wrong kinds of 
metrics, the startup was not genuinely improving. Farb was frus- 
trated in his efforts to learn from customer feedback. In every 
cycle, the type of metrics his team was focused on would change: 
one month they would look at gross usage numbers, another 
month registration numbers, and so on. Those metrics would 
go up and down seemingly on their own. He couldn’ draw clear 
cause-and-effect inferences. Prioritizing work correctly in such 
an environment is extremely challenging. | 

Farb could have asked his data analyst to investigate a 


136 THE LEAN STARTUP 


particular question. For example, when we shipped feature X, 
did it affect customer behavior? But that would have required 
tremendous time and effort. When, exactly, did feature X ship? 
Which customers were exposed to it? Was anything else launched 
around that same time? Were there seasonal factors that might 
be skewing the data? Finding these answers would have required 
parsing reams and reams of data. The answer often would come 
weeks after the question had been asked. In the meantime, the 
team would have moved on to new priorities and new questions 
that needed urgent attention. 

Compared to a lot of startups, the Grockit team had a huge 
advantage: they were tremendously disciplined. A disciplined 
team may apply the wrong methodology but can shift gears 
quickly once it discovers its error. Most important, a disciplined 
team can experiment with its own working style and draw 
meaningful conclusions. 


‘Cohorts and Split-tests 


Grockit changed the metrics they used to evaluate success in 
two ways. Instead of looking at gross metrics, Grockit switched 
to cohort-based metrics, and instead of looking for cause-and- 
effect relationships after the fact, Grockit would launch each 
new feature as a true split-test experiment. 

A split-test experiment is one in which different versions of 
a product are offered to customers at the same time. By observ- 
ing the changes in behavior between the two groups, one can 
make inferences about the impact of the different variations. 
This technique was pioneered by direct mail advertisers. For ex- 
ample, consider a company that sends customers a catalog of 
products to buy, such as Lands’ End or Crate & Barrel. If you 
wanted to test a catalog design, you could send a new version 
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of it to 50 percent of the customers and send the old standard 
catalog to the other 50 percent. To assure a scientific result, both 
catalogs would contain identical products; the only difference 
would be the changes to the design. To figure out if the new 
design was effective, all you would have to do was keep track of 
the sales figures for both groups of customers. (This technique 
is sometimes called A/B testing after the practice of assigning 
letter names to each variation.) Although split testing often is 
thought of as a marketing-specific (or even a direct marketing— 
specific) practice, Lean Startups incorporate it directly into 
product development. 

These changes led to an immediate change in Farb’s under- 
standing of the business. Split testing often uncovers surprising 
things. For example, many features that make the product bet- 
ter in the eyes of engineers and designers have no impact on 
customer behavior. This was the case at Grockit, as it has been 
in every company I have seen adopt this technique. Although 
working with split tests seems to be more difficult because it re- 
quires extra accounting and metrics to keep track of each varia- 
tion, it almost always saves tremendous amounts of time in the 
long run by eliminating work that doesn’t matter to customers. 

Split testing also helps teams refine their understanding of 
what customers want and don’t want. Grockit’s team constantly 
added new ways for their customers to interact with each other 
in the hope that those social communication tools would in- 
crease the product’s value. Inherent in those efforts was the be- 
lief that customers desired more communication during their 
studying. When split testing revealed that the extra features 
did not change customer behavior, it called that belief into 
question. 

The questioning inspired the team to seek a deeper under- 
standing of what customers really wanted. They brainstormed 
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new ideas for product experiments that might have more im- 
pact. In fact, many of these ideas were not new. They had simply 
been overlooked because the company was focused on building 
social tools. As a result, Grockit tested an intensive solo-studying 
mode, complete with quests and gamelike levels, so that students 
could have the choice of studying by themselves or with others. 
Just as in Farb’s original classroom, this proved extremely effec- 
tive. Without the discipline of split testing, the company might 
not have had this realization. In fact, over time, through dozens 
of tests, it became clear that the key to student engagement was 
to offer them a combination of social and solo features. Students 
preferred having a choice of how to study. 


Following the lean manufacturing principle of kanban, or capac- 
ity constraint, Grockit changed the product prioritization process. 
Under the new system, user stories were not considered complete 
until they led to validated learning. Thus, stories could be cata- 
loged as being in one of four states of development: in the product 
backlog, actively being built, done (feature complete from a tech- 
nical point of view), or in the process of being validated. Validated 
was defined as “knowing whether the story was a good idea to 
have been done in the first place.” This validation usually would 
come in the form of a split test showing a change in customer 
behavior but also might include customer interviews or surveys. 
The kanban rule permitted only so many stories in each of 
the four states. As stories flow from one state to the other, the 
buckets fill up. Once a bucket becomes full, it cannot accept 
more stories. Only when a story has been validated can it be re- 
moved from the kanban board. If the validation fails and it turns 
out the story is a bad idea, the relevant feature is removed from 


the product (see the chart on page 139). 
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KANBAN DIAGRAM OF WORK AS IT PROGRESSES 
FROM STAGE 10 STAGE 


(No bucket can contain more than three projects at a time.) 


BACKLOG IN PROGRESS BUILT VALIDATED 


Work on A begins. D and E are in development. F awaits validation. 


BACKLOG IN PROGRESS BUILT VALIDATED 


F is validated. D and E await validation. G, H, | are new tasks to be undertaken. B 
and C are being built. A completes development. 


BACKLOG IN PROGRESS BUILT VALIDATED 


6 
i= B= 
|— t= 


B and C have been built, but under Kanban, cannot be moved to the next bucket 
for validation until A, D, E have been validated. Work cannot begin on H and | 
until space opens up in the buckets ahead. 














I have implemented this system with several teams, and the 
initial result is always frustrating: each bucket fills up, start- 
ing with the “validated” bucket and moving on’‘to the “done” 
bucket, until it’s not possible to start any more work. Teams 
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that are used to measuring their productivity narrowly, by the 
number of stories they are delivering, feel stuck. The only way 
to start work on new features is to investigate some of the sto- 
ries that are done but haven't been validated. That often re- 
quires nonengineering efforts: talking to customers, looking at 
split-test data, and the like. 

Pretty soon everyone gets the hang of it. This progress occurs 
in fits and starts at first. Engineering may finish a big batch of 
work, followed by extensive testing and validation. As engineers 
look for ways to increase their productivity, they start to realize 
that if they include the validation exercise from the beginning, 
the whole team can be more productive. 

For example, why build a new feature that is not : part of a 
split-test experiment? It may save you time in the short run, 
but it will take more time later ‘to test, during the validation 
phase. The same logic applies to a story that an engineer doesn’t 
understand. Under the old system, he or she would just build 
it and find out later what it was for. In the new system, that 
behavior is clearly counterproductive: without a clear hypoth- 
esis, how can a story ever be validated? We saw this behavior 
at IMVU, too. I once saw a junior engineer face down a se- 
nior executive over a relatively minor change. The engineer in- 
sisted that the new feature be split-tested, just like any other. 
His peers backed him up; it was considered absolutely obvious 
that all features should be routinely tested, no matter who was 
commissioning them. (Embarrassingly, all too often I was the 
executive in question.) A solid process lays the foundation for a 
healthy culture, one where ideas are evaluated by merit and not 
by job title. 

Most important, teams working in this system begin to mea- 
sure their productivity according to validated learning, not in 
terms of the production of new features. 
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Hypothesis Testing at Grockit 


When Grockit made this transition, the results were dramatic. 
In one case, they decided to test one of their major features, 
called lazy registration, to see if it was worth the heavy invest- 
ment they were making in ongoing support. They were confi- 
dent in this feature because lazy registration is considered one 
of the design best practices for online services. In this system, 
customers do not have to register for the service up front. In- 
stead, they immediately begin using the service and are asked 
to register only after they have had a chance to experience the 
service's benefit. 

For a student, lazy registration works like this: when you 
come to the Grockit website, you're immediately placed in a 
study session with other students working on the same test. 
You don’t have to give your name, e-mail address, or credit card 
number. There is nothing to prevent you from jumping in and 
getting started immediately. For Grockit, this was essential to 
testing one of its core assumptions: that customers would be 
willing to adopt this new way of learning only if they could see 
proof that it was working early on. 

As a result of this hypothesis, Grockit’s design required that 
it manage three classes of users: unregistered guests, registered 
(trial) guests, and customers who had paid for the premium ver- 
sion of the product. This design required significant extra work 
to build and maintain: the more classes, of users there are, the 
more work is required to keep track of them, and the more mar- 
keting effort is required to create the right incentives to entice 
customers to upgrade to the next class. Grockit had undertaken 
this extra effort because lazy registration was considered an in- 
dustry best practice. 

I encouraged the team to try a simple split-test. They took 
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one cohort of customers and required that they register immedi- 
ately, based on nothing more than Grockit’s marketing materi- 
als. ‘To their surprise, this cohort’s behavior was exactly the same 
as that of the lazy registration group: they had the same rate 
of registration, activation, and subsequent retention. In other 
words, the extra effort of lazy registration was a complete waste 
even though it was considered an industry best practice. 

Even more important than reducing waste was the insight 
that this test suggested: customers were basing their decision 
about Grockit on something other than their use of the product. 

Think about this. Think about the cohort of customers who 
were required to register for the product before entering a study 
session with other students. They had very little information 
about the product, nothing more than was presented on Grack- 
its home page and registration page. By contrast, the lazy regis- 
tration group had a tremendous amount of information about 
the product because they had used it. Yet despite this informa- 
tion disparity, customer behavior was exactly the same. 

This suggested that improving Grockit’s positioning and 
marketing might have a more significant impact on attracting 
new customers than would adding new features. This was just 
the first of many important experiments Grockit was able to 
run. Since those early days, they have expanded their customer 
base dramatically: they now offer test prep for numerous stan- 
dardized tests, including the GMAT, SAT, ACT, and GRE, as 
well as online math and English courses for students in grades 7 
through 12. 

Grockit continues to evolve its process, seeking continuous 
improvement at every turn. With more than twenty employees 
in its San Francisco office, Grockit continues to operate with the 
same deliberate, disciplined approach that has been their hall- 
mark all along. They have helped close to a million students and 
are sure to help millions more. 
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THE VALUE OF THE THREE A'S 


These examples from Grockit demonstrate each of the three A’s 
of metrics: actionable, accessible, and auditable. 


Actionable 


For a report to be considered actionable, it must demonstrate 
clear cause and effect. Otherwise, it is a vanity metric. The re- 
ports that Grockit’s team began to use to judge their learning 
milestones made it extremely clear what actions would be neces- 
sary to replicate the results. 

By contrast, vanity metrics fail this criterion. Take the num- 
ber of hits to a company website. Let’s say we have 40,000 hits 
this month—a new record. What do we need to do to get more 
hits? Well, that depends. Where are the new hits coming from? 
Is it from 40,000 new customers or from one guy with an ex- 
tremely active web browser? Are the hits the result of a new mar- 
keting campaign or PR push? What is a hit, anyway? Does each 
page in the browser count as one hit, or do all the embedded 
images and multimedia content count as well? Those who have 
sat in a meeting debating the units of measurement in a report 
will recognize this problem. 

Vanity metrics wreak havoc because they prey on a weakness 
of the human mind. In my experience, when the numbers go 
up, people think the improvement was caused by their actions, 
by whatever they were working on at the time. That is why it’s 
so common to have a meeting in which marketing thinks the 
numbers went up because of a new PR or marketing effort and 
engineering thinks the better numbers are the result of the new 
features it added. Finding out what is actually going on is ex- 
tremely costly, and so most managers simply move on, doing the 
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best they can to form their own judgment on the basis of their 
experience and the collective intelligence in the room. 

Unfortunately, when the numbers go down, it results in a 
very different reaction: now it’s somebody else’s fault. Thus, 
most team members or departments live in a world where their 
department is constantly making things better, only to have 
their hard work sabotaged by other departments that just don’t. 
get it. Is it any wonder these departments develop their own dis- 
tinct language, jargon, culture, and defense mechanisms against 
the bozos working down the hall? 

Actionable metrics are the antidote to this problem. When 
cause and effect is clearly understood, people are better able to 
learn from their actions. Human beings are innately talented 
learners when given a clear and objective assessment. 


Accessible 


All too many reports are not understood by the employees and 
managers who are supposed to use them to guide their decision 
making. Unfortunately, most managers do not respond to this 
complexity by working hand in hand with the data warehousing 
team to simplify the reports so that they can understand them 
better. Departments too often spend their energy learning how 
to use data to get what they want rather than as genuine feed- 
back to guide their future actions. 

There is an antidote to this misuse of data. First, make the 
reports as simple as possible so that everyone understands them. 
Remember the saying “Metrics are people, too.” The easiest 
way to make reports comprehensible is to use tangible, concrete 
units. What is a website hit? Nobody is really sure, but everyone 
knows what a person visiting the website is: one can practically 
picture those people sitting at their computers. 

This is why cohort-based reports are the gold standard of 
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learning metrics: they turn complex actions into people-based 
reports. Each cohort analysis says: among the people who used 
our product in this period, here’s how many of them exhibited 
each of the behaviors we care about. In the IMVU example, 
we saw four behaviors: downloading the product, logging into 
the product from one’s computer, engaging in a chat with other 
customers, and upgrading to the paid version of the product. 
In other words, the report deals with people and their actions, 
which are far more useful than piles of data points. For example, 
think about how hard it would have been to tell if IMVU was 
being successful if we had reported only on the total number of 
person-to-person conversations. Let's say we have 10,000 con- 
versations in a period. Is that good? Is that one person being 
very, very social, or is it 10,000 people each trying the product 
one time and then giving up? There’s no way to know without 
creating a more detailed report. 

As the gross numbers get larger, accessibility becomes more 
and more important. It is hard to visualize what it means if the 
number of website hits goes down from 250,000 in one month 
to 200,000 the next month, but most people understand im- 
mediately what it means to lose 50,000 customers. That's prac- 
tically a whole stadium full of people who are abandoning the 
product. | 

Accessibility also refers to widespread access to the reports. 
Grockit did this especially well. Every day their system automat- 
ically generated a document containing the latest data for every 
single one of their split-test experiments and other leap-of-faith 
metrics. This document was mailed to every employee of the 
company: they all always had a fresh copy in their e-mail in- 
boxes. The reports were well laid out and easy to read, with each 
experiment and its results explained in plain English. 

Another way to make reports accessible is to use a technique 
we developed at IMVU. Instead of housing the analytics or data 
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in a separate system, our reporting data and its infrastructure 
were considered part of the product itself and were owned by 
the product development team. The reports were available on 
our website, accessible to anyone with an employee account. 

Each employee could log in to the system at any time, 
choose from a list of all current and past experiments, and see 
a simple one-page summary of the results. Over time, those 
one-page summaries became the de facto standard for settling 
product arguments throughout the organization. When peo- 
ple needed evidence to support something they had learned, 
they would bring a printout with them to the relevant meet- 
ing, confident that everyone they showed it to would under- 
stand its meaning. | 


Auditable 


When informed that their pet project is a failure, most of us 
are tempted to blame the messenger, the data, the manager, the 
gods, or anything else we can think of. That’s why the third A 
of good metrics, “auditable,” is so essential. We must ensure that 
the data is credible to employees. 

The employees at IMVU would brandish one-page reports 
to demonstrate what they had learned to settle arguments, but 
the process often wasn't so smooth. Most of the time, when a 
manager, developer, or team was confronted with results that 
would kill a pet project, the loser of the argument would chal- 
lenge the veracity of the data. 

Such challenges are more common than most managers 
would admit, and unfortunately, most data reporting systems 
are not designed to answer them successfully. Sometimes this 
is the result of a well-intentioned but misplaced desire to pro- 
tect the privacy of customers. More often, the lack of such sup- 
porting documentation is simply a matter of neglect. Most data 
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reporting systems are not built by product development teams, 
whose job is to prioritize and build product features. They are 
built by business managers and analysts. Managers who must 
use these systems can only check to see if the reports are mutu- 
ally consistent. They all too often lack a way to test if the data is 
consistent with reality. 

The solution? First, remember that “Metrics are people, too.” 
We need to be able to test the data by hand, in the messy real 
world, by talking to customers. This is the only way to be able to 
check if the reports contain true facts. Managers need the ability 
to spot check the data with real customers. It also has a second 
benefit: systems that provide this level of auditability give man- 
agers and entrepreneurs the opportunity to gain insights into 
why customers are behaving the way the data indicate. 

Second, those building reports must make sure the mecha- 
nisms that generate the reports are not too complex. Whenever 
possible, reports should be drawn directly from the master data, 
rather than from an intermediate system, which reduces oppor- 
tunities for error. I have noticed that every time a team has one 
of its judgments or assumptions overturned as a result of a tech- 
nical problem with the data, its confidence, morale, and disci- 
pline are undermined. 


oO 0 0 


When we watch entrepreneurs succeed in the mythmaking 
world of Hollywood, books, and magazines, the story is always 
structured the same way. First, we see the plucky protagonist 
having an epiphany, hatching a great new idea. We learn about 
his or her character and personality, how he or she came to be 
in the right place at the right time, and how he or she took the 
dramatic leap to start a business. 

Then the photo montage begins. It’s usually short, just a 
few minutes of time-lapse photography or narrative. We see the 


148 THE LEAN STARTUP 


protagonist building a team, maybe working in a lab, writing on 
whiteboards, closing sales, pounding on a few keyboards. At the 
end of the montage, the founders are successful, and the story 
can move on to more interesting fare: how to split the spoils 
of their success, who will appear on magazine covers, who sues 
whom, and implications for the future. | 

Unfortunately, the real work that determines the success of 
startups happens during the photo montage. It doesn’t make 
the cut in terms of the big story because it is too boring. Only 
5 percent of entrepreneurship is the big idea, the business 
model, the whiteboard strategizing, and the splitting up of the 
spoils. The other 95 percent is the gritty work that is measured 
by innovation accounting: product prioritization decisions, 
deciding which customers to target or listen to, and having 
the courage to subject a grand vision to constant testing and 
feedback. | 

One decision stands out above all others as the most diff- 
cult, the most time-consuming, and the biggest source of waste 
for most startups. We all must face this fundamental test: decid- 
ing when to pivot and when to persevere. To understand what 
happens during the photo montage, we have to understand how 
to pivot, and that is the subject of Chapter 8. 


a 
PIVOT (OR PEROEVERE) 


a entrepreneur eventually faces an overriding challenge in de- 
veloping a successful product: deciding when to pivot and 
when to persevere. Everything that has been discussed so far is 
a prelude to a seemingly simple question: are we making suf- 
ficient progress to believe that our original strategic hypothesis 
is correct, or do we need to make a major change? That change 
is called a pivot: a structured course correction designed to test 
a new fundamental hypothesis about the product, strategy, and 
engine of growth. 

Because of the scientific methodology that underlies the Lean 
Startup, there is often a misconception that it offers a rigid clini- 
cal formula for making pivot or persevere decisions. This is not 
true. There is no way to remove the human element—vision, 
intuition, judgment—from the practice of entrepreneurship, 
nor would that be desirable. | 

My goal in advocating a scientific approach to the creation of 
startups is to channel human creativity into its most productive 
form, and there is no bigger destroyer of creative potential than 
the misguided decision to persevere. Companies that cannot 
bring themselves to pivot to a new direction on the basis of feed- 
back from the marketplace can get stuck in the land of the living 
dead, neither growing enough nor dying, consuming resources 
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and commitment from employees and other stakeholders but 
not moving ahead. 

There is good news about our reliance on judgment, though. 
We are able to learn, we are innately creative, and we have a re- 
markable ability to see the signal in the noise. In fact, we are so 
good at this that sometimes we see signals that aren't there. The 
heart of the scientific method is the realization that although 
human judgment may be faulty, we can improve our judgment 
by subjecting our theories to repeated testing. 

Startup productivity is not about cranking out more widgets 
or features. It is about aligning our efforts with a business and 
product that are working to create value and drive growth. In 
other words, successful pivots put us on a path toward growing 
a sustainable business. 


INNOVATION ACCOUNTING LEADS 10 FASTER PIVOTS 


To see this process in action, meet David Binetti, the CEO of 
Votizen. David has had a long career helping to bring the Amer- 
ican political process into the twenty-first century. In the early 
1990s, he helped build USA.gov, the first portal for the federal 
government. He’s also experienced some classic startup failures. 
When it came time to build Votizen, David was determined to 
avoid betting the farm on his vision. 

David wanted to tackle the problem of civic participation in 
the political process. His first product concept was a social net- 
work of verified voters, a place where people passionate about 
civic causes could get together, share ideas, and recruit their 
friends. David built his first minimum viable product for just - 
over $1,200 in about three months and launched it. 

David wasn't building something that zobody wanted. In fact, 
from its earliest days, Votizen was able to attract early adopters 
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who loved the core concept. Like all entrepreneurs, David had 
to refine his product and business model. What made David’s 
challenge especially hard was that he had to make those pivots 
in the face of moderate success. 

David's initial concept involved four big leaps of faith: 


1. Customers would be interested enough in the social net- 
work to sign up. (Registration) 

2. Votizen would be able to verify them as registered voters. 
(Activation) 

3. Customers who were verified voters would engage with 
the site’s activism tools over time. (Retention) 

4, Engaged customers would tell their friends about the ser- 
vice and recruit them into civic causes. (Referral) 


Three months and $1,200 later, David’s firsts MVP was in 
customers hands. In the initial cohorts, 5 percent signed up for 
the service and 17 percent verified their registered voter status 
(see the chart below). The numbers were so low that there wasn’t 
enough data to tell what sort of engagement or referral would 
occur. It was time to start iterating. 


David spent the next two months and another $5,000 split test- 
ing new product features, messaging, and improving the product's 
design to make it easier to use. Those tests showed dramatic im- 
provements, going from a 5 percent registration rate to 17 percent 
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and from a 17 percent activation rate to over 90 percent. Such is 
the power of split testing. This optimization gave David a criti- 
cal mass of customers with which to measure the next two leaps 
of faith. However, as shown in the chart below, those numbers 
proved to be even more discouraging: David achieved a referral 
rate of only 4 percent and a retention rate of 5 percent. 


| INITIAL MP =| AFTER OPTIMIZATION 


Registration 
Activation 
Retention 


Referral Too low 


David knew he had to do more development and testing. For 
the next three months he continued to optimize, split test, and 
refine his pitch. He talked to customers, held focus groups, and 
did countless A/B experiments. As was explained in Chapter 7, 
in a split test, different versions of a product are offered to dif- 
ferent customers at the same time. By observing the changes 
in behavior between the two groups, one can make inferences 
about the impact of the different variations. As shown in the 
chart below, the referral rate nudged up slightly to 6 percent and 
the retention rate went up to 8 percent. A disappointed David 
had spent eight months and $20,000 to build a product that 
wasn't living up to the growth model he'd hoped for. 





BEFORE OPTIMIZATION AFTER OPTIMIZATION 
Registration 17% 


Retention 5% 8% 
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David faced the difficult challenge of deciding whether to 
pivot or persevere. This is one of the hardest decisions entre- 
preneurs face. The goal of creating learning milestones is not to 
make the decision easy; it is to make sure that there is relevant 
data in the room when it comes time to decide. 

Remember, at this point David has had many customer con- 
versations. He has plenty of learning that he can use to ratio- 
nalize the failure he has experienced with the current product. 
That’s exactly what many entrepreneurs do. In Silicon Valley, 
we call this experience getting stuck in the land of the living 
dead. It happens when a company has achieved a modicum of 
success—just enough to stay alive—but is not living up to the 
expectations of its founders and investors. Such companies are 
a terrible drain of human energy. Out of loyalty, the employees 
and founders don’t want to give in; they feel that success might 
be just around the corner. 

David had two advantages that helped him avoid this fate: 


1. Despite being committed to a significant vision, he had 
done his best to launch early and iterate. Thus, he was 
facing a pivot or persevere moment just eight months 
into the life of his company. The more money, time, and 
creative energy that has been sunk into an idea, the harder 
it is to pivot. David had done well to avoid that trap. 

2. David had identified his leap-of-faith questions explicitly 
at the outset and, more important, had made quantita- 
tive predictions about each of them. It would not have 
been difficult for him to declare success retroactively from 
that initial venture. After all, some of his metrics, such as 
activation, were doing quite well. In terms of gross met- 
rics such as total usage, the company had positive growth. 
It is only because David focused on actionable metrics 
for each of his leap-of-faith questions that he was able to 
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accept that his company was failing. In addition, because 
David had not wasted energy on premature PR, he was 
able to make this determination without public embar- 
rassment or distraction. 


Failure is a prerequisite to learning. The problem with the 
notion of shipping a product and then seeing what happens is 
that you are guaranteed to succeed—at seeing what happens. 
But then what? As soon as you have a handful of customers, 
youre likely to have five opinions about what to do next. Which 
should you listen to? 

Votizen’s results were okay, but they were not ag6H enough. 
David felt that although his optimization was improving the 
metrics, they were not trending toward a model that would sus- 
tain the business overall. But like all good entrepreneurs, he did 
not give up prematurely. David decided to pivot and test a new 
hypothesis. A pivot requires that we keep one foot rooted in 
what we've learned so far, while making a fundamental change 
in strategy in order to seek even greater validated learning. In 
this case, David’s direct contact with customers proved essential. 

He had heard three recurring bits of feedback in his testing: 


1. “I always wanted to get more involved; this makes it so 
much easier.” 

2. “The fact that you prove I’m a voter matters.” 

3. “There’s no one here. What's the point of coming back?”’ 


David decided to undertake what I call a zoom-in pivot, re- 
focusing the product on what previously had been considered 
just one feature of a larger whole. Think of the customer com- 
ments above: customers like the concept, they like the voter 
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registration technology, but they aren't getting value out of the 
social networking part of the product. 

David decided to change Votizen into a product called 
@2gov, a “social lobbying platform.” Rather than get custom- 
ers integrated in a civic social network, @2gov allows them 
to contact their elected representatives quickly and easily via 
existing social networks such as Twitter. The customer engages 
digitally, but @2gov translates that digital contact into paper 
form. Members of Congress receive old-fashioned printed let- 
ters and petitions as a result. In other words, @2gov translates 
the high-tech world of its customers into the low-tech world 
of politics. 

@2gov had a slightly different set of leap-of-faith questions 
to answer. It still depended on customers signing up, verifying 
their voter status, and referring their friends, but the growth 
model changed. Instead of relying on an engagement-driven 
business (“sticky” growth), @2gov was more transactional. Da- 
vid’s hypothesis was that passionate activists would be willing to 
pay money to have @2gov facilitate contacts on behalf of voters 
who cared about their issues. 

David's new MVP took four months and another $30,000. 
Hed now spent a grand total of $50,000 and worked for twelve 
months. But the results from his next round of testing were 
dramatic: registration rate 42 percent, activation 83 percent, 
retention 21 percent, and referral a whopping 54 percent. 
However, the number of activists willing to pay was less than 
1 percent. The value of each transaction was far too low to sus- 
tain a profitable business even after David had done his best to 
optimize It. 

Before we get to David’s next pivot, notice how convinc- 
ingly he was able to demonstrate validated learning. He hoped 
that with this new product, he would be able to improve his 
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leap-of-faith metrics dramatically, and he did (see the chart 
below). 


acne 
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He did this not by working harder but by working smarter, 
taking his product development resources and applying them . 
to a new and different product. Compared with the previous 
four months of optimization, the new four months of pivot- 
ing had resulted in a dramatically higher return on investment, 
but David was still stuck in an age-old entrepreneurial trap. His 
metrics and product were improving, but not fast enough. 

David pivoted again. This time, rather than rely on activists 
to pay money to drive contacts, he went to large organizations, 
professional fund-raisers, and big companies, which all have a 
professional or business interest in political campaigning. The 
companies seemed extremely eager to use and pay for David's 
service, and David quickly signed letters of intent to build the 
functionality they needed. In this pivot, David did what I call a 
- customer segment pivot, keeping the functionality of the product 
the same but changing the audience focus. He focused on who 
pays: from consumers to businesses and nonprofit organizations. 
In other words, David went from being a business-to-consumer 
(B2C) company to being a business-to-business (B2B) com- 
pany. In the process he changed his planned growth model, as 
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well to one where he would be able to fund growth out of the 
profits generated from each B2B sale. | 

Three months later, David had built the functionality he 
had promised, based on those early letters of intent. But when 
he went back to companies to collect his checks, he discovered 
more problems. Company after company procrastinated, de- 
layed, and ultimately passed up the opportunity. Although they 
had been excited enough to sign a letter of intent, closing a real 
sale was much more difficult. It turned out that those compa- 
nies were not early adopters. 

On the basis of the letters of intent, David had increased his 
head count, taking on additional sales staff and engineers in antic- 
ipation of having to service higher-margin business-to-business 
accounts. When the sales didn’t materialize, the whole team had 
to work harder to try to find revenue elsewhere. Yet no matter 
how many sales calls they went on and no matter how much 
optimization they did to the product, the model wasn’t working. 
Returning to his leap-of-faith questions, David concluded that 
the results refuted his business-to-business hypothesis, and so he 
decided to pivot once again. 

All this time, David was learning and gaining feedback from 
his potential customers, but he was in an unsustainable situa- 
tion. You can’t pay staff with what you've learned, and raising 
money at that juncture would have escalated the problem. Rais- 
ing money without early traction is not a certain thing. If he had 
been able to raise money, he could have kept the company going 
but would have been pouring money into a value-destroying en- 
gine of growth. He would be in a high-pressure situation: use 
investors cash to make the engine of growth work or risk having 
to shut-down the company (or be replaced). 

David decided to reduce staff and pivot again, this time at- 
tempting what I call a platform pivot. Instead of selling an ap- 


plication to one customer at a time, David envisioned a new 
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growth model inspired by Google’s AdWords platform. He built 


a self-serve sales platform where anyone could become a cus- 
tomer with just a credit card. Thus, no matter what cause you 
were passionate about, you could go to @2gov's website and 
@2gov would help you find new people to get involved. As al- 
ways, the new people were verified registered voters, and so their 
opinions carried weight with elected officials. 

The new product took only one additional month to build 
and immediately showed results: 51 percent sign-up rate, 92 
percent activation rate, 28 percent retention rate, 64 percent re- 
ferral rate (see the chart below). Most important, 11 percent of 
these customers were willing to pay 20 cents per message. Most 
important, this was the beginning of an actual growth model 
that could work. Receiving 20 cents per message might not 
sound like much, but the high referral rate meant that @2gov 
could grow its traffic without spending significant marketing 
money (this is the viral engine of growth). 


err 
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Votizen’s story exhibits some common patterns. One of the 
most important to note is the acceleration of MVPs. The first 
MVP took eight months, the next four months, then three, then 
one. Each time David was able to validate or refute his next hy- 
pothesis faster than before. 
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How can one explain this acceleration? It is tempting to credit 
it to the product development work that had been going on. 
Many features had been created, and with them a fair amount 
of infrastructure. Therefore, each time the company pivoted, it 
didn’t have to start from scratch. But this is not the whole story. 
For one thing, much of the product had to be discarded be- 
tween pivots. Worse, the product that remained was classified 
asa legacy product, one that was no longer suited to the goals 
of the company. As is usually the case, the effort required to 
reform a legacy product took extra work. Counteracting these 
forces were the hard-won lessons David had learned through 
each milestone. Votizen accelerated its MVP process because 
it was learning critical things about its customers, market, and 
strategy. | 

Today, two years after its inception, Votizen is doing well. 
They recently raised $1.5 million from Facebook’s initial inves- 
tor Peter Thiel, one of the very few consumer Internet invest- 
* ments he has made in recent years. Votizen’s system now can 
process voter identity in real time for forty-seven states represent- 
ing 94 percent of the U.S. population and has delivered tens of 
thousands of messages to Congress. The Startup Visa campaign 
used Votizen’s tools to introduce the Startup Visa Act (S.565), 
which is the first legislation introduced into the Senate solely as 
a result of social lobbying. These activities have attracted the at- 
tention of established Washington consultants who are seeking 
to employ Votizen’s tools in future political campaigns. 

David Binetti sums up his experience building a Lean 
Startup: 


In 2003 I started a company in roughly the same space 
as I’m in today. I had roughly the same domain expertise 
and industry credibility, fresh off the USA.gov success. 
But back then my company was a total failure (despite 
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consuming significantly greater investment), while now 
I have a business making money and closing deals. 
Back then I did the traditional linear product develop- 
ment model, releasing an amazing product (it really was) 
after 12 months of development, only to find that no 
one would buy it. This time I produced four versions in 
twelve weeks and generated my first sale relatively soon 
after that. And it isn’t just market timing—two other 
companies that launched in a similar space in 2003 sub- 
sequently sold for tens of millions of dollars, and others 
in 2010 followed a linear model straight to the dead pool. 


A STARTUP'S RUNWAY IS THE NUMBER OF PIVOTS IT CAN STILL MAKE 


Seasoned entrepreneurs often speak of the runway that their 
_ startup has left: the amount of time remaining in which a startup 
must either achieve lift-off or fail. This usually is defined as the ° 
remaining cash in the bank divided by the monthly burn rate, or 
net drain on that account balance. For example, a startup with 
$1 million in the bank that is spending $100,000 per month 
has a projected runway of ten months. 

When startups start to run low on cash, they can extend 
the runway two ways: by cutting costs or by raising additional 
funds. But when entrepreneurs cut costs indiscriminately, they 
are as liable to cut the costs that are allowing the company to get 
through its Build-Measure-Learn feedback loop as they are to 
cut waste. If the cuts result in a slowdown to this feedback loop, 
all they have accomplished is to help the startup go out of busi- 
ness more slowly. | 

The true measure of runway is how many pivots a startup has 
left: the number of opportunities it has to make a fundamental 
change to its business strategy. Measuring runway through the 
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lens of pivots rather than that of time suggests another way to 
extend that runway: get to each pivot faster. In other words, the 
startup has to find ways to achieve the same amount of validated 
learning at lower cost or in a shorter time. All the techniques in 
the Lean Startup model that have been discussed so far have this 
as their overarching goal. 


PIVOTS REQUIRE COURAGE 


Ask most entrepreneurs who have decided to pivot and they will 
tell you that they wish they had made the decision sooner. I be- 
lieve there are three reasons why this happens. 

First, vanity metrics can allow entrepreneurs to form false 
conclusions and live in their own private reality. This is particu- 
larly damaging to the decision to pivot because it robs teams of 
the belief that it is necessary to change. When people are forced 
to change against their better judgment, the process is harder, 
takes longer, and leads to a less decisive outcome. 

Second, when an entrepreneur has an unclear hypothesis, 
its almost impossible to experience complete failure, and with- 
out failure there is usually no impetus to embark on the radical 
change a pivot requires. As I mentioned earlier, the failure of the 
“launch it and see what happens” approach should now be evi- 
dent: you will always succeed—in seeing what happens. Except 
in rare cases, the early results will be ambiguous, and you won't 
know whether to pivot or persevere, whether to change direc- 
tion or stay the course. 

Third, many entrepreneurs are afraid. Acknowledging failure 
can lead to dangerously low morale. Most entrepreneurs’ biggest 
fear is not that their vision will prove to be wrong. More terrifying 
is the thought that the vision might be deemed wrong without 
having been given a real chance to prove itself. This fear drives 
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much of the resistance to the minimum viable product, split test- 
ing, and other techniques to test hypotheses. Ironically, this fear 
drives up the risk because testing doesn’t occur until the vision 
is fully represented. However, by that time it is often too late to 
pivot because funding is running out. To avoid this fate, entre- 
preneurs need to face their fears and be willing to fail, often in a 
public way. In fact, entrepreneurs who have a high profile, either 
because of personal fame or because they are operating as part of a 
famous brand, face an extreme version of this problem. 

A new startup in Silicon Valley called Path was started by 
experienced entrepreneurs: Dave Morin, who previously had 
overseen Facebook's platform initiative; Dustin Mierau, prod- 
uct designer and cocreator of Macster; and Shawn Fanning of 
Napster fame. They decided to release a minimum viable prod- 
uct in 2010. Because of the high-profile nature of its founders, 
the MVP attracted significant press attention, especially from 
technology and startup blogs. Unfortunately, their product was | 
not targeted at technology early adopters, and as a result, the 
early blogger reaction was quite negative. (Many entrepreneurs 
fail to launch because they are afraid of this kind of reaction, 
worrying that it will harm the morale of the entire company. 
The allure of positive press, especially in our “home” industry, 
is quite strong.) 

Luckily, the Path team had the courage to ignore this fear 
and focus on what their customers said. As a result, they were 
able to get essential early feedback from actual customers. Path's 
goal is to create a more personal social network that maintains 
its quality over time. Many people have had the experience of 
being overconnected on existing social networks, sharing with 
past coworkers, high school friends, relatives, and colleagues. 
Such broad groups make it hard to share intimate moments. 
. Path took an unusual approach. For example, it limited the 
number of connections to fifty, based on brain research by the 
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anthropologist Robin Dunbar at Oxford. His research suggests 
that fifty is roughly the number of personal relationships in any 
person's life at any given time. 

For members of the tech press (and many tech early adopt- 
ers) this “artificial” constraint on the number of connections was 
anathema. They routinely use new social networking products 
with thousands of connections. Fifty seemed way too small. As 
a result, Path endured a lot of public criticism, which was hard 
to ignore. But customers flocked to the platform, and their feed- 
back was decidedly different from the negativity in the press. 
Customers liked the intimate moments and consistently wanted 
features that were not on the original product road map, such as 
the ability to share how friends’ pictures made them feel and the 
ability to share “video moments.” 

Dave Morin summed up his experience this way: 


The reality of our team and our backgrounds built up a 
massive wall of expectations. I don’t think it would have 
mattered what we would have released; we would have 
been met with expectations that are hard to live up to. 
But to us it just meant we needed to get our product 
and our vision out into the market broadly in order to 
get feedback and to begin iteration. We humbly test our 
theories and our approach to see what the market thinks. 
Listen to feedback honestly. And continue to innovate in 
the directions we think will create meaning in the world. 


Path’s story is just beginning, but already their courage in 
facing down critics is paying off. If and when they need to pivot, 
they won't be hampered by fear. They recently raised $8.5 mil- 
lion in venture capital in a round led by Kleiner Perkins Caufield 
& Byers. In doing so, Path reportedly turned down an acquisi- 
tion offer for $100 million from Google.” 
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THE PIVOT OR PERSEVERE MEETING 


The decision to pivot requires a clear-eyed and objective 
mind-set. We've discussed the telltale signs of the need to pivot: 
the decreasing effectiveness of product experiments and the gen- 
eral feeling that product development should be more produc- 
tive. Whenever you see those symptoms, consider a pivot. 

The decision to pivot is emotionally charged for any startup 
and has to be addressed in a structured way. One way to miti- 
gate this challenge is to schedule the meeting in advance. I rec- 
ommend that every startup have a regular “pivot or persevere” 
meeting. In my experience, less than a few weeks between meet- 
ings is too often and more than a few months is too infrequent. 
However, each startup needs to find its own pace. 

Each pivot or persevere meeting requires the participation of 
both the product development and business leadership teams. 
At IMVU, we also added the perspectives of outside advisers 
who could help us see past our preconceptions and interpret 
data in new ways. The product development team must bring 
a complete report of the results of its product optimization ef- 
forts over time (not just the past period) as well as a comparison 
of how those results stack up against expectations (again, over 
time). The business leadership should bring detailed accounts of 
their conversations with current and potential customers. 

Let’s take a look at this process in action in a dramatic pivot 
done by a company called Wealthfront. That company was 
founded in 2007 by Dan Carroll and added Andy Rachleff as 
CEO shortly thereafter. Andy is a well-known figure in Silicon 
Valley: he is a cofounder and former general partner of the ven- 
ture capital firm Benchmark Capital and is on the faculty of the 
Stanford Graduate School of Business, where he teaches a vari- 
ety of courses on technology entrepreneurship. I first met Andy 
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when he commissioned a case study on IMVU to teach his stu- 
dents about the process we had used to build the company. 

Wealthfront’s mission is to disrupt the mutual fund indus- 
try by bringing greater transparency, access, and value to retail 
investors. What makes Wealthfront’s story unusual, however, is 
not where it is today but how it began: as an online game. 

In Wealthfront’s original incarnation it was called kaChing 
and was conceived as a kind of fantasy league for amateur inves- 
tors. It allowed anyone to open a virtual trading account and 
build a portfolio that was based on real market data without hav- 
ing to invest real money. The idea was to identify diamonds in 
the rough: amateur traders who lacked the resources to become 
fund managers but who possessed market insight. Wealthfront’s 
founders did not want to be in the online gaming business per 
se; kaChing was part of a sophisticated strategy in the service of 
their larger vision. Any student of disruptive innovation would 
have looked on approvingly: they were following that system 
perfectly by initially serving customers who were unable to par- 
ticipate in the mainstream market. Over time, they believed, the 
product would become more and more sophisticated, eventually 
allowing users to serve (and disrupt) existing professional fund 
managers. 

To identify the best amateur trading savants, Wealthfront 
built sophisticated technology to rate the skill of each fund 
manager, using techniques employed by the most sophisticated 
evaluators of money managers, the premier U.S. university en- 
dowments. Those methods allowed them to evaluate not just the 
returns the managers generated but also the amount of risk they 
had taken along with how consistent they performed relative to 
their declared investment strategy. Thus, fund managers who 
achieved great returns through reckless gambles (i.e., investments 
outside their area of expertise) would be ranked lower than those 


who had figured out how to beat the market through skill. 
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With its kaChing game, Wealthfront hoped to test two 


leap-of-faith assumptions: 


1. A significant percentage of the game players would dem- 
onstrate enough talent as virtual fund managers to prove 
themselves suitable to become managers of real assets (the 
value hypothesis). 

2. The game would grow using the viral engine of growth 

_and generate value using a freemium business model. The 
game was free to play, but the team hoped that a per- 
centage of the players would realize that they were lousy 
traders and therefore want to convert to paying customers 
once Wealthfront started offering real asset management 
services (the growth hypothesis). 


kaChing was a huge early success, attracting more than 
450,000 gamers in its initial launch. By now, you should be 
suspicious of this kind of vanity metric. Many less disciplined 
companies would have celebrated that success and felt their 
future was secure, but Wealthfront had identified its assump- 
tions clearly and was able to think more rigorously. By the time 
Wealthfront was ready to launch its paid financial product, 
only seven amateur managers had qualified as worthy of man- 
aging other people’s money, far less than the ideal model had 
anticipated. After the paid product launched, they were able to 
measure the conversion rate of gamers into paying customers. 
Here too the numbers were discouraging: the conversion rate 
was close to zero. Their model had predicted that hundreds of 
customers would sign up, but only fourteen did. 

The team worked valiantly to find ways to improve the prod- 
uct, but none showed any particular promise. It was time for a 
pivot or persevere meeting. 


Pivot (or Persevere) 6f 


If the data we have discussed so far was all that was available 
at that critical meeting, Wealthfront would have been in trou- 
ble. They would have known that their current strategy wasn’t 
working but not what to do to fix it. That is why it was critical 
that they followed the recommendation earlier in this chapter to 
investigate alternative possibilities. In this case, Wealthfront had 
pursued two important lines of inquiry. 

The first was a series of conversations with professional 
money managers, beginning with John Powers, the head of 
Stanford University’s endowment, who reacted surprisingly pos- 
itively. Wealthfront’s strategy was premised on the assumption 
that professional money managers would be reluctant to join 
the system because the increased transparency would threaten 
their sense of authority. Powers had no such concerns. CEO 
Andy Rachleff then began a series of conversations with other 
professional investment managers and brought the results back 
to the company. His insights were as follows: 


1. Successful professional money managers felt they had 
nothing to fear from transparency, since they believed it 
would validate their skills. 

2. Money managers faced significant challenges in managing 
and scaling their own businesses. They were hampered by 
the difficulty of servicing their own accounts and there- 
fore had to require high minimum investments as a way 
to screen new clients. 


The second problem was so severe that Wealthfront was 
fielding cold calls from professional managers asking out of the 
blue to join the platform. These were classic early adopters who 
had the vision to see past the current product to something they 
could use to achieve a competitive advantage. 
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The second critical qualitative information came out of con- 
versations with consumers. It turned out that they found the 
blending of virtual and real portfolio management on the ka- 
Ching website confusing. Far from being a clever way of acquir- 
ing customers, the freemium strategy was getting in the way by 
promoting confusion about the company’s positioning. 

This data informed the pivot or persevere meeting. With ev- 
eryone present, the team debated what to do with its future. 
The current strategy wasn't working, but many employees were 
nervous about abandoning the online game. After all, it was an 
important part of what they had signed on to build. They had 
invested significant time and energy building and supporting 
those customers. It was painful—as it always is—to realize that 
that energy had been wasted. 

Wealthfront decided it could not persevere as it existed. The 
company chose instead to celebrate what it had learned. If it had 
not launched its current product, the team never would have 
learned what it needed to know to pivot. In fact, the experience 
taught them something essential about their vision. As Andy 
says, “What we really wanted to change was not who manages 
the money but who has access to the best possible talent. Wed 
originally thought we'd need to build a significant business with 
amateur managers to get professionals to come on board, but 
fortunately it turns out that wasn’t necessary.” 

The company pivoted, abandoning the gaming custom- 
ers altogether and focusing on providing a service that allowed 
customers to invest with professional managers. On the surface, 
the pivot seems quite dramatic in that the company changed 
its positioning, its name, and its partner strategy. It even jetti- 
soned a large proportion of the features it had built. But at its 
core, a surprising amount stayed the same. The most valuable 
work the company had done was building technology to evaluate 
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managers effectiveness, and this became the kernel around which 
the new business was built. This is also common with pivots; it 
is not necessary to throw out everything that came before and 
_ start over. Instead, it’s about repurposing what has been built and 
what has been learned to find a more positive direction. 

Today, Wealthfront is prospering as a result of its pivot, with 
over $180 million invested on the platform and more than forty 
professional managers.’ It recently was named one of Fast Com- 
pany’s ten most innovative companies in finance.* The company 
continues to operate with agility, scaling in line with the growth 
principles outlined in Chapter 12. Wealthfront is also a leading 
advocate of the development technique known as continuous 
deployment, which we'll discuss in Chapter 9. 


FAILURE T0 PIVOT 


The decision to pivot is so difficult that many companies fail to 
make it. I wish I could say that every time I was confronted with 
the need to pivot, I handled it well, but this is far from true. I 
remember one failure to pivot especially well. 

A few years after IMVU’s founding, the company was having 
tremendous success. The business had grown to over $1 million 
per month in revenue; we had created more than twenty million 
avatars for our customers. We managed to raise significant new 
rounds of financing, and like the global economy, we were rid- 
ing high. But danger lurked around the corner. 

Unknowingly, we had fallen into a classic startup trap. We 
had been so successful with our early efforts that we were ignor- 
ing the principles behind them. As a result, we missed the need 
to pivot even as it stared us in the face. 

We had built an organization that excelled at the kinds of 
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activities described in earlier chapters: creating minimum viable 
products to test new ideas and running experiments to tune the 
engine of growth. Before we had begun to enjoy success, many 
people had advised against our “low-quality” minimum viable 
product and experimental approach, urging us to slow down. 
They wanted us to do things right and focus on quality instead 
of speed. We ignored that advice, mostly because we wanted to 
claim the advantages of speed. After our approach was vindi- 
cated, the advice we received changed. Now most of the advice 
we heard was that “you can’t argue with success,” urging us to stay 
the course. We liked this advice better, but it was equally wrong. 

Remember that the rationale for building low-quality MVPs 
is that developing any features beyond what early adopters re- 
quire is a form of waste. However, the logic of this takes you 
only so far. Once you have found success with early adopters, 
you want to sell to mainstream customers. Mainstream custom- 
ers have different requirements and are much more demanding. 

The kind of pivot we needed is called a customer segment 
pivot. In this pivot, the company realizes that the product it’s 
building solves a real problem for real customers but that they 
are not the customers it originally planned to serve. In other 
words, the product hypothesis is confirmed only partially. (This 
chapter described such a pivot in the Votizen story, above.) 

A customer segment pivot is an especially tricky pivot to ex- 
ecute because, as we learned the hard way at IMVU, the very 
actions that made us successful with early adopters were diametri- 
cally opposed to the actions we'd have to master to be successful 
with mainstream customers. We lacked a clear understanding of 
how our engine of growth operated. We had begun to trust our 
vanity metrics. We had stopped using learning milestones to hold 
-ourselves accountable. Instead, it was much more convenient to 
focus on the ever-larger gross metrics that were so exciting: break- 
ing new records in signing up paying customers and active users, 
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monitoring our customer retention rate—you name it. Under 
the surface, it should have been clear that our efforts at tuning the 
engine were reaching diminishing returns, the classic sign of the 
need to pivot. 

For example, we spent months trying to improve the product's 
activation rate (the rate at which new customers become active 
consumers of the product), which remained stubbornly low. We 
did countless experiments: usability improvements, new persua- 
sion techniques, incentive programs, customer quests, and other 
game-like features. Individually, many of these new features and 
new marketing tools were successful. We measured them rigor- 
ously, using A/B experimentation. But taken in aggregate, over 
the course of many months, we were seeing negligible changes in 
the overall drivers of our engine of growth. Even our activation 
rate, which had been the center of our focus, edged up only a few 
percentage points. 

We ignored the signs because the company was still growing, 
delivering month after month of “up and to the right” results. 
But we were quickly exhausting our early adopter market. It was 
getting harder and harder to find customers we could acquire at 
the prices we were accustomed to paying. As we drove our mar- 
keting team to find more customers, they were forced to reach 
out more to mainstream customers, but mainstream customers 
are less forgiving of an early product. The activation and mon- 
etization rates of new customers started to go down, driving up 
the cost of acquiring new customers. Pretty soon, our growth 
was flatlining and our engine sputtered and stalled. 

It took us far too long to make the changes necessary to fix 
this situation. As with all pivots, we had to get back to basics 
and start the innovation accounting cycle over. It felt like the 
company’s second founding. We had gotten really good at op- 
_timizing, tuning, and iterating, but in the process we had lost 
sight of the purpose of those activities: testing a clear hypothesis 
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in the service of the company’s vision. Instead, we were chasing 
growth, revenue, and profits wherever we could find them. 

‘We needed to reacquaint ourselves with our new mainstream 
customers. Our interaction designers led the way by developing 
a clear customer archetype that was based on extensive in-person 
conversations and observation. Next, we needed to invest heavily 
in a major product overhaul designed to make the product dra- 
matically easier to use. Because of our overfocus on fine-tuning, 
we had stopped making large investments like these, preferring 
to invest in lower-risk and lower-yield testing experiments. 

However, investing in quality, design, and larger projects 
did not require that we abandon our experimental roots. On 
the contrary, once we realized our mistake and executed the 
pivot, those skills served us well. We created a sandbox for ex- 
perimentation like the one described in Chapter 12 and had a 
cross-functional team work exclusively on this major redesign. 
As they built, they continuously tested their new design head 
to head against the old one. Initially, the new design performed 
worse than the old one, as is usually the case. It lacked the fea- 
tures and functionality of the old design and had many new 
mistakes as well. But the team relentlessly improved the design 
until, months later, it performed better. This new design laid the 
foundation for our future growth. 

This foundation has paid off handsomely. By 2009, revenue 
had more than doubled to over $25 million annually. But we 
might have enjoyed that success earlier if we had pivoted sooner.” 


A CATALOG OF PIVOTS 


Pivots come in different flavors. The word pivot sometimes is 
used incorrectly as a synonym for change. A pivot is a special 
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kind of change designed to test a new fundamental hypothesis 
about the product, business model, and engine of growth. 


Zoom-in Pivot 


In this case, what previously was considered a single feature in 
a product becomes the whole product. This is the type of pivot 
Votizen made when it pivoted away from a full social network 
and toward a simple voter contact product. 


Zoom-out Pivot 


In the reverse situation, sometimes a single feature is insufficient 
to support a whole product. In this type of pivot, what was con- 
sidered the whole product becomes a single feature of a much 
larger product. 


Customer Segment Pivot 


In this pivot, the company realizes that the product it is build- 
ing solves a real problem for real customers but that they are not 
the type of customers it originally planned to serve. In other 
words, the product hypothesis is partially confirmed, solving 
the right problem, but for a different customer than originally 
anticipated. 


Customer Need Pivot 


As a result of getting to know customers extremely well, it some- 
times becomes clear that the problem we're trying to solve for 
them is not very important. However, because of this customer 
intimacy, we often discover other related problems that are 
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important and can be solved by our team. In many cases, these 
related problems may require little more than repositioning the 
existing product. In other cases, it may require a completely new 
product. Again, this a case where the product hypothesis is par- 
tially confirmed; the target customer has a problem worth solv- 
ing, just not the one that was originally anticipated. 

A famous example is the chain Potbelly Sandwich Shop, 
which today has over two hundred stores. It began as an antique 
store in 1977; the owners started to sell sandwiches as a way to 
bolster traffic to their stores. Pretty soon they had pivoted their 
way into an entirely different line of business. 


Platform Pivot 


A platform pivot refers to a change from an application to a 
platform or vice versa. Most commonly, startups that aspire to 
create a new platform begin life by selling a single application, 
the so-called killer app, for their platform. Only later does the 
platform emerge as a vehicle for third parties to leverage as a way 
to create their own related products. However, this order is not 
always set in stone, and some companies have to execute this 
pivot multiple times. 


Business Architecture Pivot 


This pivot borrows a concept from Geoffrey Moore, who ob- 
served that companies generally follow one of two major busi- 
ness architectures: high margin, low volume (complex systems 
model) or low margin, high volume (volume operations model). 
The former commonly is associated with business to business 
(B2B) or enterprise sales cycles, and the latter with consumer 
products (there are notable exceptions). In a business architec- 
ture pivot, a startup switches architectures. Some companies 
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change from high margin, low volume by going mass market 
(e.g., Google's search “appliance”); others, originally designed 
for the mass market, turned out to require long and expensive 
sales cycles. 


Value Capture Pivot 


There are many ways to capture the value a company creates. 
These methods are referred to commonly as monetization or 
revenue models. These terms are much too limiting. Implicit 
in the idea of monetization is that it is a separate “feature” of a 
product that can be added or removed at will. In reality, captur- 
ing value is an intrinsic part of the product hypothesis. Often, 
changes to the way a company captures value can have far- 
reaching consequences for the rest of the business, product, and 
marketing strategies. 


Engine of Growth Pivot 


As we'll see in Chapter 10, there are three primary engines of 
growth that power startups: the viral, sticky, and paid growth 
models. In this type of pivot, a company changes its growth 
' strategy to seek faster or more profitable growth. Commonly 
but not always, the engine of growth also requires a change in 
the way value is captured. 


Channel Pivot 


In traditional sales terminology, the mechanism by which a com- 
pany delivers its product to customers is called the sales channel 
or distribution channel. For example, consumer packaged goods 
are sold in a grocery store, cars are sold in dealerships, and much 
enterprise software is sold (with extensive customization) by 
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consulting and professional services firms. Often, the require- 
ments of the channel determine the price, features, and com- 
petitive landscape of a product. A channel pivot is a recognition 
that the same basic solution could be delivered through a dif- 
ferent channel with greater effectiveness. Whenever a company 
abandons a previously complex sales process to “sell direct” to its 
end users, a channel pivot is in progress. 

It is precisely because of its destructive effect on sales chan- 
nels that the Internet has had such a disruptive influence in in- 
dustries that previously required complex sales and distribution 
channels, such as newspaper, magazine, and book publishing. 


Technology Pivot 


Occasionally, a company discovers a way to achieve the same 
solution by using a completely different technology. Technol- 
ogy pivots are much more common in established businesses. 
In other words, they are a sustaining innovation, an incremental 
improvement designed to appeal to and retain an existing cus- 
tomer base. Established companies excel at this kind of pivot 
because so much is not changing. The customer segment is the 
same, the customer’s problem is the same, the value-capture 
model is the same, and the channel partners are the same. The 
only question is whether the new technology can provide su- 
perior price and/or performance compared with the existing 


technology. 


A PIVOT IS A STRATEGIC HYPOTHESIS 


Although the pivots identified above will be familiar to stu- 
dents of business strategy, the ability to pivot is no substitute for 
sound strategic thinking. The problem with providing famous 
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examples of pivots is that most people are familiar only with 
the successful end strategies of famous companies. Most readers 
know that Southwest or Walmart is an example of a low-cost 
disruption in their markets, that Microsoft an example of a plat- 
form monopoly, and that Starbucks has leveraged a powerful 
premium brand. What is generally less well known are the piv- 
ots that were required to discover those strategies. Companies 
have a strong incentive to align their PR stories around the he- 
roic founder and make it seem that their success was the inevi- 
table result of a good idea. 

Thus, although startups often pivot into a strategy that seems 
similar to that of a successful company, it is important not to 
put too much stock in these analogies. It’s extremely difficult to 
know if the analogy has been drawn properly. Have we copied 
the essential features or just superficial ones? Will what worked 
in that industry work in ours? Will what has worked in the past 
work today? A pivot is better understood as a new strategic hy- 
pothesis that will require a new minimum viable product to test. 

Pivots are a permanent fact of life for any growing business. 
Even after a company achieves initial success, it must continue 
to pivot. Those familiar with the technology life cycle ideas of 
theorists such as Geoffrey Moore know certain later-stage pivots 
by the names he has given them: the Chasm, the Tornado, the 
Bowling Alley. Readers of the disruptive innovation literature 
spearheaded by Harvard’s Clayton Christensen will be familiar 
with established companies that fail to pivot when they should. 
The critical skill for managers today is to match those theories 
to their present situation so that they apply the right advice at 
the right time. 

Modern managers cannot have escaped the deluge of recent 
books calling on them to adapt, change, reinvent, or upend their 
existing businesses. Many of the works in this category are long 
on exhortations and short on specifics. 
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A pivot is not just an exhortation to change. Remember, it 
is a special kind of structured change designed to test a new 
fundamental hypothesis about the product, business model, and 
engine of growth. It is the heart of the Lean Startup method. It 
is what makes the companies that follow Lean Startup resilient 
in the face of mistakes: if we take a wrong turn, we have the 

tools we need to realize it and the agility to find another path. 


O00 


In Part Two, we have looked at a startup idea from its initial 
leaps of faith, tested it with a minimum viable product, used 
innovation accounting and actionable metrics to evaluate the 
results, and made the decision to pivot or persevere. 

I have treated these subjects in great detail to prepare for 
what comes next. On the page, these processes may seem clini- 
cal, slow, and simple. In the real world, something different is 
needed. We have learned to steer when moving slowly. Now we 
must learn to race. Laying a solid foundation is only the first - 
step toward our real destination: acceleration. 


Part Ihree 


ALLELERATE 


Start Your Engines 


Most of the decisions startups face are not clear-cut. How often 
should you release a product? Is there a reason to release weekly 
rather than daily or quarterly or annually? Product releases incur 
overhead, and so from an efficiency point of view, releasing 
often leaves less time to devote to building the product. How- 
ever, waiting too long to release can lead to the ultimate waste: 
- making something that nobody wants. 

How much time and energy should companies invest in in- 
frastructure and planning early on in anticipation of success? 
Spend too much and you waste precious time that could have 
been spent learning. Spend too little and you may fail to take 
advantage of early success and cede market leadership to a fast 
follower. 

What should employees spend their days doing? How do 
we hold people accountable for learning at an organizational 
level? Traditional departments create incentive structures that 
keep people focused on excellence in their specialties: market- 
ing, sales, product development. But what if the company’s best 
interests are served by cross-functional collaboration? Startups 
need organizational structures that combat the extreme uncer- 
tainty that is a startup’s chief enemy. 

The lean manufacturing movement faced similar questions 
on the factory floor. Their answers are relevant for startups as 
well, with some modifications. 

The critical first question for any lean transformation is: 
which activities create value and which are a form of waste? 
Once you understand this distinction, you can begin using lean 
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techniques to drive out waste and increase the efficiency of the 
value-creating activities. For these techniques to be used in a 
startup, they must be adapted to the unique circumstances of 
entrepreneurship. Recall from Chapter 3 that value in a startup 
is not the creation of stuff, but rather validated learning about 
how to build a sustainable business. What products do custom- 
ers really want? How will our business grow? Who is our cus- 
tomer? Which customers should we listen to and which should 
we ignore? These are the questions that need answering as 
quickly as possible to maximize a startup’s chances of success. 
That is what creates value for a startup. 

In Part Three, we will develop techniques that allow Lean 
Startups to grow without sacrificing the speed and agility that 
are the lifeblood of every startup. Contrary to common belief, 
lethargy and bureaucracy are not the inevitable fate of com- 
panies as they achieve maturity. I believe that with the proper 
foundation, Lean Startups can grow to become lean enterprises 
that maintain their agility, learning orientation, and culture of 
innovation even as they scale. 

In Chapter 9, we will see how Lean Startups take advan- 
tage of the counterintuitive power of small batches. Just as lean 
manufacturing has pursued a just-in-time approach to build- 
ing products, reducing the need for in-process inventory, Lean 
Startups practice just-in-time scalability, conducting product 
experiments without making massive up-front investments in 
planning and design. 

Chapter 10 will explore the metrics startups should use to 
understand their growth as they add new customers and dis- 
cover new markets. Sustainable growth follows one of three 
engines of growth: paid, viral, or sticky. By identifying which 
engine of growth a startup is using, it can then direct energy 
where it will be most effective in growing the business. Each en- 
gine requires a focus on unique metrics to evaluate the success of 
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new products and prioritize new experiments. When used with 
the innovation accounting method described in Part ‘Two, these 
metrics allow startups to figure out when their growth is at risk 
of running out and pivot accordingly. 

Chapter 11 shows how to build an adaptive organization by 
investing in the right amount of process to keep teams nimble as 
they grow. We will see how techniques from the tool kit of lean 
manufacturing, such as the Five Whys, help startup teams grow 
without becoming bureaucratic or dysfunctional. We also will 
see how lean disciplines set the stage for a startup to transition 
into an established company driven by operational excellence. 

In Chapter 12, we'll come full circle. As startups grow into 
established companies, they face the same pressures that make 
it necessary for today’s enterprises to find new ways to invest 
in disruptive innovation. In fact, we'll see that an advantage 
of a successful startup’s rapid growth is that the company can 
keep its entrepreneurial DNA even as it matures. Today's com- 
panies must learn to master a management portfolio of sustain- 
able and disruptive innovation. It is an obsolete view that sees 
startups as going through discrete phases that leave earlier kinds 
of work—such as innovation—behind. Rather, modern compa- 
nies must excel at doing multiple kinds of work in parallel. To 
do so, we'll explore techniques for incubating innovation teams 
within the context of an established company. 

I have included an epilogue called “Waste Not” in which I 
consider some of the broader implications of the success of the 
Lean Startup movement, place it in historical context (including 
cautionary lessons from past movements), and make suggestions 
for its future direction. | 


BATCH 


nthe book Lean /hinking, James Womack and Daniel Jones tre-, 

count a story of stuffing newsletters into envelopes with the 
assistance of one of the author’s two young children. Every en- 
velope had to be addressed, stamped, filled with a letter, and 
sealed. The daughters, age six and nine, knew how they should 
go about completing the project: “Daddy, first you should fold 
all of the newsletters. Then you should attach the seal. Then 
you should put on the stamps.” Their father wanted to do it the 
counterintuitive way: complete each envelope one at a time. 
They—like most of us—thought that was backward, explain- 
ing to him “that wouldn't be efficient!” He and his daughters 
each took half the envelopes and competed to see who would 
finish first. 

The father won the race, and not just because he is an adult. 
It happened because the one envelope at a time approach is a 
faster way of getting the job done even though it seems inef- 
ficient. This has been confirmed in many studies, including one 
that was recorded on video.’ 

The one envelope at a time approach is called “single-piece 
flow” in lean manufacturing. It works because of the surpris- 
ing power of small batches. When we do work that proceeds in 
stages, the “batch size” refers to how much work moves from 
one stage to the next at a time. For example, if we were stuffing 
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one hundred envelopes, the intuitive way to do it—folding one 
hundred letters at a time—would have a batch size of one hun- 
dred. Single-piece flow is so named because it has a batch size 
of-one. 

Why does stuffing one envelope at a time get the job done 
faster even though it seems like it would be slower? Because our 
intuition doesn’t take into account the extra time required to 
sort, stack, and move around the large piles of half-complete 
envelopes when it’s done the other way.” It seems more efficient 
to repeat the same task over and over, in part because we expect 
that we will get better at this simple task the more we do it. Un- 
fortunately, in process-oriented work like this, individual per- 
formance is not nearly as important as the overall performance 
of the system. 

Even if the amount of time that each process took was exactly 
the same, the small batch production approach still would be 
superior, and for even more counterintuitive reasons. For exam- 
ple, imagine that the letters didn’t fit in the envelopes. With the 
large-batch approach, we wouldn't find that out until nearly the 
end. With small batches, we'd know almost immediately. What 
if the envelopes are defective and wont seal? In the large-batch 
approach, wed have to unstuff all the envelopes, get new ones, 
and restuff them. In the small-batch approach, wed find this 
out immediately and have no rework required. 

All these issues are visible in a process as simple as stuffing 
envelopes, but they are of real and much greater consequence in 
the work of every company, large or small. The small-batch ap- 
proach produces a finished product every few seconds, whereas 
the large-batch approach must deliver all the products at once, 
at the end. Imagine what this might look like if the time horizon 
was hours, days, or weeks. What if it turns out that the custom- 
ers have decided they don’t want the product? Which process 
would allow a company to find this out sooner? 
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Lean manufacturers discovered the benefits of small batches 
decades ago. In the post-World War II economy, Japanese car- 
makers such as Toyota could not compete with huge American 
factories that used the latest mass production techniques. Fol- 
lowing the intuitively efficient way of building, mass production 
factories built cars by using ever-larger batch sizes. They would 
spend huge amounts of money buying machines that could pro- 
duce car parts by the tens, hundreds, or thousands. By keeping — 
those machines running at peak speed, they could drive down 
the unit cost of each part and produce cars that were incredibly 
inexpensive so long as they were completely uniform. 

The Japanese car market was far too small for companies 
such as Toyota to employ those economies of scale; thus, Japa- 
nese companies faced intense pressure from mass production. 
Also, in the war-ravaged Japanese economy, capital was not 
available for massive investments in large machines. 

It was against this backdrop that innovators such as Taiichi 
Ohno, Shigeo Shingo, and others found a way to succeed by 
using small batches. Instead of buying large specialized ma- 
chines that could produce thousands of parts at a time, Toyota 
used smaller general-purpose machines that could produce a 
wide variety of parts in small batches. This required figuring out 
ways to reconfigure each machine rapidly to make the right part 
at the right time. By focusing on this “changeover time,” Toyota 
was able to produce entire automobiles by using small batches 
throughout the process. 

This rapid changing of machines was no easy feat. As in any 
lean transformation, existing systems and tools often need to be 
reinvented to support working in smaller batches. Shigeo Shingo 
created the concept of SMED (Single-Minute Exchange of Die) 
in order to enable a smaller batch size of work in early Toyota 
factories. He was so relentless in rethinking the way machines 
were operated that he was able to reduce changeover times that 
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previously took hours to less than ten minutes. He did this, not 
by asking workers to work faster, but by reimagining and re- 
structuring the work that needed to be done. Every investment 
in better tools and process had a corresponding benefit in terms 
of shrinking the batch size of work. 

Because of its smaller batch size, Toyota was able to produce 
a much greater diversity of products. It was no longer necessary 
that each product be exactly the same to gain the economies of 
scale that powered mass production. Thus, Toyota could serve 
its smaller, more fragmented markets and still compete with the 
mass producers. Over time, that capability allowed Toyota to 
move successfully into larger and larger markets until it became 
the world’s largest automaker in 2008. 

The biggest advantage of working in small batches is that 
quality problems can be identified much sooner. This is the ori- 
gin of Toyota's famous andon cord, which allows any worker to 
ask for help as soon as they notice any problem, such as a defect 
in a physical part, stopping the entire production line if it can- 
not be corrected immediately. This is another very counterintui- 
tive practice. An assembly line works best when it is functioning 
smoothly, rolling car after car off the end of the line. The andon 
cord can interrupt this careful flow as the line is halted repeat- 
edly. However, the benefits of finding and fixing problems faster 
outweigh this cost. This process of continuously driving out de- 
fects has been a win-win for Toyota and its customers. It is the 
root cause of Toyota's historic high quality ratings and low costs. 


OMALL BATCHES IN ENTREPRENEURSHIP 


When I teach entrepreneurs this method, I often begin with 
stories about manufacturing. Before long, I can see the ques- 
tioning looks: what does this have to do with my startup? The 
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theory that is the foundation of Toyota’s success can be used to 
dramatically improve the speed at which startups find validated 
learning. | 

Toyota discovered that small batches made their factories 
more efficient. In contrast, in the Lean Startup the goal is not to 
produce more stuff efficiently. It is to—as quickly as possible— 
learn how to build a sustainable business. 

Think back to the example of envelope stuffing. What if 
it turns out that the customer doesn’t want the product we're 
building? Although this is never good news for an entrepreneur, 
finding out sooner is much better than finding out later. Work- 
ing in small batches ensures that a startup can minimize the ex- 
penditure of time, money, and effort that ultimately turns out 
to have been wasted. 


Small Batches at IMVU 


At IMVU, we applied these lessons from manufacturing to the 
way we work. Normally, new versions of products like ours are 
released to customers on a monthly, quarterly, or yearly cycle. 

Take a look at your cell phone. Odds are, it is not the very 
first version of its kind. Even innovative companies such as 
Apple produce a new version of their flagship phones about 
once a year. Bundled up in that product release are dozens of 
new features (at the release of iPhone 4, Apple boasted more 
than 1,500 changes). 

Ironically, many high-tech products are manufactured in ad- 
vanced facilities that follow the latest in lean thinking, including 
small batches and single-piece flow. However, the process that is 
used to design the product is stuck in the era of mass produc- 
tion. Think of all the changes that are made to a product such 
as the iPhone; all 1,500 of them are released to customers in one 
giant batch. 
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Behind the scenes, in the development and design of the 
product itself, large batches are still the rule. The work that goes 
into the development of a new product proceeds on a virtual as- 
sembly line. Product managers figure out what features are likely 
to please customers; product designers then figure out how 
those features should look and feel. These designs are passed to 
engineering, which builds something new or modifies an exist- 
ing product and, once this is done, hands it off to somebody 
responsible for verifying that the new product works the way the 
product managers and designers intended. For a product such as 
the iPhone, these internal handoffs may happen on a monthly 
or quarterly basis. 

Think back one more time to the envelope-stuffing exercise. 
What is the most efficient way to do this work? 

At IMVU, we attempted to design, develop, and ship our 
new features one at a time, taking advantage of the power of 
small batches. Here’s what it looked like. 

Instead of working in separate departments, engineers and 
designers would work together side by side on one feature at a 
time. Whenever that feature was ready to be tested with custom- 
ers, they immediately would release a new version of the prod- 
uct, which would go live on our website for a relatively small 
number of people. The team would be able immediately to as- 
sess the impact of their work, evaluate its effect on customers, | 
and decide what to do next. For tiny changes, the whole process 
might be repeated several times per day. In fact, in the aggregate, 
IMVU makes about fifty changes to its product (on average) 
every single day. 

Just as with the Toyota Production System, the key to being 
able to operate this quickly is to check for defects immedi- 
ately, thus preventing bigger problems later. For example, we 
had an extensive set of automated tests that assured that after 
every change our product still worked as designed: Let’s say an 
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engineer accidentally removed an important feature, such as the 
checkout button on one of our e-commerce pages. Without this 
button, customers no longer could buy anything from IMVU. 
It's as if our business instantly became a hobby. Analogously to 
the Toyota andon cord, IMVU used an elaborate set of defense 
mechanisms that prevented engineers from accidentally break- 
ing something important. 

We called this our product’s immune system because those 
automatic protections went beyond checking that the prod- 
uct behaved as expected. We also continuously monitored the 
health of our business itself so that mistakes were found and 
removed automatically. 

Going back to our business-to-hobby example of the miss- 
ing checkout button, let’s make the problem a little more 
interesting. Imagine that instead of removing the button alto- 
gether, an engineer makes a mistake and changes the button’s 
color so that it is now white on a white background. From the 
point of view of automated functional tests, the button is still 
there and everything is working normally; from the customer's 
point of view, the button is gone, and so nobody can buy any- 
thing. This class of problems is hard to detect solely with auto- 
mation but is still catastrophic from a business point of view. 
At IMVU, our immune system is programmed to detect these 
business consequences and automatically invoke our equiva- 
lent of the andon cord. 

When our immune system detects a problem, a number of 
things happen immediately: 


1. The defective change is removed immediately and 
automatically. | 

2. Everyone on the relevant team is notified of the problem. 

3. The team is blocked from introducing any further changes, 
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preventing the problem from being compounded by future 
mistakes . . . 

4, ... until the root cause of the problem is found and fixed. 
(This root cause analysis is discussed in greater detail in 


Chapter 11.) 


At IMVU, we called this continuous deployment, and even in 
the fast-moving world of software development it is still consid- 
ered controversial.’ As the Lean Startup movement has gained 
traction, it has come to be embraced by more and more startups, 
even those that operate mission-critical applications. Among the 
most cutting edge examples is Wealthfront, whose pivot was de- 
scribed in Chapter 8. The company practices true continuous 
deployment—including more than a dozen releases to custom- 
ers every day—in an SEC-regulated environment.‘ 


Continuous Deployment Beyond Software 


When I tell this story to people who work in a slower-moving 
industry, they think I am describing something futuristic. But 
increasingly, more and more industries are seeing their design 
process accelerated by the same underlying forces that make this 
kind of rapid iteration possible in the software industry. There 
are three ways in which this is happening: 


1. Hardware becoming software. Think about what has 
happened in consumer electronics. The latest phones and tab- 
let computers are little more than a screen connected to the 
Internet. Almost all of their value is determined by their soft- 
ware. Even old-school products such as automobiles are see- 
ing ever-larger parts of their value being generated by the 
software they carry inside, which controls everything from the 
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entertainment system to tuning the engine to controlling the 
brakes. What can be built out of software can be modified much 


faster than a physical or mechanical device can. 


2. Fast production changes. Because of the success of the 
lean manufacturing movement, many assembly lines are set up 
to allow each new product that comes off the line to be custom- 
ized completely without sacrificing quality or cost-effectiveness. 
Historically, this has been used to offer the customer many 
choices of product, but in the future, this capability will allow 
the designers of products to get much faster feedback about new 
versions. When the design changes, there is no excess inventory 
of the old version to slow things down. Since machines are de- 
signed for rapid changeovers, as soon as the new design is ready, 
new versions can be produced quickly. 


3. 3D printing and rapid prototyping tools. As just one 
_ example, most products and parts that are made out of plastic 
today are mass produced using a technique called injection mold- 
ing. This process is extremely expensive and time-consuming to 
set up, but once it is up and running, it can reproduce hundreds 
of thousands of identical individual items at an extremely low 
cost. It is a classic large-batch production process. This has put 
entrepreneurs who want to develop a new physical product at a 
disadvantage, since in general only large companies can afford 
these large production runs for a new product. However, new 
technologies are allowing entrepreneurs to build small batches of 
products that are of the same quality as products made with injec- 
tion molding, but at much lower cost and much, much faster. 


The essential lesson is not that everyone should be shipping fifty 
times per day but that by reducing batch size, we can get through 
the Build-Measure-Learn feedback loop more quickly than our 
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competitors can. The ability to learn faster from customers is the 
essential competitive advantage that startups must possess. 


SMALL BATCHES IN ACTION 


To see this process in action, let me introduce you to a company 
in Boise, Idaho, called SGW Designworks. SGW’s specialty is 
rapid production techniques for physical products. Many of its 
clients are startups. 

SGW Designworks was engaged by a client who had been 
asked by a military customer to build a complex field x-ray sys- 
tem to detect explosives and other destructive devices at border 
crossings and in war zones. 

Conceptually, the system consisted of an advanced head unit 
that read x-ray film, multiple x-ray film panels, and the frame- 
work to hold the panels while the film was being exposed. The 
client already had the technology for the x-ray panels and the 
head unit, but to make the product work in rugged military set- 
tings, SGW needed to design and deliver the supporting struc- 
ture that would make the technology usable in the field. The 
framework had to be stable to ensure a quality x-ray image, du- 
rable enough for use in a war zone, easy to deploy with minimal 
training, and small enough to collapse into a backpack. 

This is precisely the kind of product we are accustomed to 
thinking takes months or years to develop, yet new techniques 
are shrinking that time line. SGW immediately began to gener- 
ate the visual prototypes by using 3D computer-aided design 
(CAD) software. The 3D models served as a rapid communica- 
tion tool between the client and the SGW team to make early 
design decisions. 

The team and client settled on a design that used an ad- 
vanced locking hinge to provide the collapsibility required 
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without compromising stability. The design also integrated a 
suction cup/pump mechanism to allow for fast, repeatable at- 
tachment to the x-ray panels. Sounds complicated, right? 

Three days later, the SGW team delivered the first physical 
prototypes to the client. The prototypes were machined out of 
aluminum directly from the 3D model, using a technique called 
computer numerical control (CNC) and were hand assembled 
by the SGW team. 

The client immediately took the prototypes to its military 
contact for review. The general concept was accepted with a 
number of minor design modifications. In the next five days, 
another full cycle of design iteration, prototyping, and design 
review was completed by the client and SGW. The first produc- 
tion run of forty completed units was ready for delivery three 
and a half weeks after the initiation of the development project. 

SGW realized that this was a winning model because feed- 
back on design decisions was nearly instantaneous. The team 
used the same process to design and deliver eight products, serv- 
ing a wide range of functions, in a twelve-month period. Half 
of those products are generating revenue today, and the rest are 
awaiting initial orders, all thanks to the power of working in 


small batches. 
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Small Batches in Education 


Not every type of product—as it exists today—allows for de- 
sign change in small batches. But that is no excuse for stick- 
ing to outdated methods. A significant amount of work may be 
needed to enable innovators to experiment in small batches. As 
was pointed out in Chapter 2, for established companies look- 
ing to accelerate their innovation teams, building this platform 
for experimentation is the responsibility of senior management. 

Imagine that you are a schoolteacher in charge of teach- 
ing math to middle school students. Although you may teach 
concepts in small batches, one day at a time, your overall cur- 
riculum cannot change very often. Because you must set up the 
curriculum in advance and teach the same concepts in the same 
order to every student in the classroom, you can try a new cur- 
riculum at most only once a year. 

How could a math teacher experiment with small batches? 
Under the current large-batch system for educating students, 
it would be quite difficult; our current educational system was 
designed in the era of mass production and uses large batches 
extensively. 

A new breed of startups is working hard to change all that. In 
School. of One, students have daily “playlists” of their learning 
tasks that are attuned to each student’s learning needs, based on 
that student's readiness and learning style. For example, Julia is 
way ahead of grade level in math and learns best in small groups, 
so her playlist might include three or four videos matched to her 
aptitude level, a thirty-minute one-on-one tutoring session with 
her teacher, and a small group activity in which she works on a 
math puzzle with three peers at similar aptitude levels. There are 
assessments built into each activity so that data can be fed back 
to the teacher to choose appropriate tasks for the next playlist. 
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This data can be aggregated across classes, schools, or even whole 
districts. 7 

Now imagine trying to experiment with a curriculum by 
using a tool such as School of One. Each student is working 
at his or her own pace. Let's say you are a teacher who has a 
new sequence in mind for how math concepts should be taught. 
You can see immediately the impact of the change on those of 
your students who are at that point in the curriculum. If you 
judge it to be a good change, you could roll it out immediately 
for every single student; when they get to that part of the cur- 
riculum, they will get the new sequence automatically. In other 
words, tools like School of One enable teachers to work in much 
smaller batches, to the benefit of their students. (And, as tools 
reach wide-scale adoption, successful experiments by individual 
teachers can be rolled out district-, city-, or even nationwide.) 
This approach is having an impact and earning accolades. Time 
magazine recently included School of One in its “most innova- 
tive ideas” list; it was the only educational organization to make 


the list.° 


THE LARGE-BATCH DEATH SPIRAL 


Small batches pose a challenge to managers steeped in traditional 
notions of productivity and progress, because they believe that 
functional specialization is more efficient for expert workers. 

Imagine youre a product designer overseeing a new prod- 
uct and you need to produce thirty individual design drawings. 
It probably seems that the most efficient way to work is in se- 
clusion, by yourself, producing the designs one by one. Then, 
when youre done with all of them, you pass the drawings on to 
the engineering team and let them work. In other words, you 
work in large batches. 
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From the point of view of individual efficiency, working in 
large batches makes sense. It also has other benefits: it promotes 
skill building, makes it easier to hold individual contributors 
accountable, and, most important, allows experts to work with- 
out interruption. At least that’s the theory. Unfortunately, reality 
seldom works out that way. 

Consider our hypothetical example. After passing thirty. de- 
sign drawings to engineering, the designer is free to turn his or 
her attention to the next project. But remember the problems 
that came up during the envelope-stuffing exercise. What hap- 
pens when engineering has questions about how the drawings 
are supposed to work? What if some of the drawings are unclear? 
What if something goes wrong when engineering attempts to 
use the drawings? 

These problems inevitably turn into interruptions for the 
designer, and now those interruptions are interfering with the 
next large batch the designer is supposed to be working on. If 
the drawings need to be redone, the engineers may become idle 
while they wait for the rework to be completed. If the designer is 
not available, the engineers may have to redo the designs them- 
selves. This is why so few products are actually built the way 
they are designed. 

When I work with product managers and designers in com- 
panies that use large batches, I often discover that they have to 
redo their work five or six times for every release. One product 
manager I worked with was so inundated with interruptions 
that he took to coming into the office in the middle of the night 
so that he could work uninterrupted. When I suggested that he 
try switching the work process from large-batch to single-piece 
flow, he refused—because that would be inefficient! So strong 
is the instinct to work in large batches, that even when a large- 
batch system is malfunctioning, we have a tendency to blame 
ourselves. 
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Large batches tend to grow over time. Because moving the 
batch forward often results in additional work, rework, de- 
lays, and interruptions, everyone has an incentive to do work 
in ever-larger batches, trying to minimize this overhead. This 
is called the large-batch death spiral because, unlike in manufac- 
turing, there are no physical limits on the maximum size of a 
batch.° It is possible for batch size to keep growing and growing. 
Eventually, one batch will become the highest-priority project, 
a “bet the company” new version of the product, because the 
company has taken such a long time since the last release. But 
now the managers are incentivized to increase batch size rather 
than ship the product. In light of how long the product has 
been in development, why not fix one more bug or add one 
more feature? Who really wants to be the manager who risked 
the success of this huge release by failing to address a potentially 
critical flaw? | 

I worked at a company that entered this death spiral. We 
had been working for months on a new version of a really cool 
product. The original version had been years in the making, and 
expectations for the next release were incredibly high. But the 
longer we worked, the more afraid we became of how customers 
would react when they finally saw the new version. As our plans 
became more ambitious, so too did the number of bugs, con- 
flicts, and problems we had to deal with. Pretty soon we got into 
a situation in which we could not ship anything. Our launch 
date seemed to recede into the distance. The more work we got 
done, the more work we had to do. The lack of ability to ship 
eventually precipitated a crisis and a change of management, all 
because of the trap of large batches. 

These misconceptions about batch size are incredibly com- 
mon. Hospital pharmacies often deliver big batches of medi- 
cations to patient floors once a day because it’s efficient (a 
single trip, right?). But many of those meds get sent back to the 
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pharmacy when a patient’s orders have changed or the patient 
is moved or discharged, causing the pharmacy staff to do lots 
of rework and reprocessing (or trashing) of meds. Delivering 
smaller batches every four hours reduces the total workload for 
the pharmacy and ensures that the right meds are at the right 
place when needed. 

Hospital lab blood collections often are done in hourly 
batches; phlebotomists collect blood for an hour from multiple 
patients and then send or take all the samples to the lab. This 
adds to turnaround time for test results and can harm test qual- 
ity. It has become common for hospitals to bring small batches 
(two patients) or a single-patient flow of specimens to the lab 
even if they have to hire an extra phlebotomist or two to do so, 
because the total system cost is lower.’ 


PULL, DON'T PUSH 


Let's say you are out for a drive, pondering the merits of small 
batches, and find yourself accidentally putting a dent in your 
new 2011 blue Toyota Camry. You take it into the dealership 
for repair and wait to hear the bad news. The repair technician 
tells you that you need to have the bumper replaced. He goes to 
check their inventory levels and tells you he has a new bumper 
in stock and they can complete your repair immediately. This 
is good news for everyone—you because you get your car back 
sooner and the dealership because they have a happy customer 
and dont run the risk of your taking the car somewhere else 
for repair. Also, they don’t have to store your car or give you a 
loaner while they wait for the part to come in. 

In traditional mass production, the way to avoid stockouts— 
not having the product the customer wants—is to keep a large 
inventory of spares just in case. It may be that the blue 2011 
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Camry bumper is quite popular, but what about last year’s 
model or the model from five years ago? The more inventory you 
keep, the greater the likelihood you will have the right product 
in stock for every customer. But large inventories are expensive 
because they have to be transported, stored, and tracked. What 
if the 2011 bumper turns out to have a defect? All the spares in 
all the warehouses instantly become waste. 

Lean production solves the problem of stockouts with a 
technique called pull. When you bring a car into the dealership 
for repair, one blue 2011 Camry bumper gets used. This creates 
a “hole” in the dealer’s inventory, which automatically causes a 
signal to be sent to a local restocking facility called the Toyota 
Parts Distribution Center (PDC). The PDC sends the dealer 
a new bumper, which creates another hole in inventory. This 
sends a similar signal to a regional warehouse called the Toyota 
Parts Redistribution Center (PRC), where all parts suppliers 
ship their products. That warehouse signals the factory where 
the bumpers are made to produce one more bumper, which is 
manufactured and shipped to the PRC. 

The ideal goal is to achieve small batches all the way down to 
single-piece flow along the entire supply chain. Each step in the 
line pulls the parts it needs from the previous step. This is the 
famous Toyota just-in-time production method.° 

When companies switch to this kind of production, their 
warehouses immediately shrink, as the amount of just-in-case 
inventory [called work-in-progress (WIP) inventory] is reduced 
dramatically. This almost magical shrinkage of WIP is where 
lean manufacturing gets its name. It’s as if the whole supply 
chain suddenly went on a diet. 

Startups struggle to see their wore-pionesr a inven- 
tory. When factories have excess WIP, it literally piles up on 
the factory floor. Because most startup work is intangible, it's 
not nearly as visible. For example, all the work that goes into 
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designing the minimum viable product is—until the moment 
that product is shipped—just WIP inventory. Incomplete de- 
signs, not-yet-validated assumptions, and most business plans 
are WIP. Almost every Lean Startup technique we've discussed 
so far works its magic in two ways: by converting push methods 
to pull and reducing batch size. Both have the net effect of re- 
ducing WIP. 

In manufacturing, pull is used primarily to make sure pro- 
duction processes are tuned to levels of customer demand. 
Without this, factories can wind up making much more—or 
much less—of a product than customers really want. How- 
ever, applying this approach to developing new products is not 
straightforward. Some people misunderstand the Lean Startup 
model as simply applying pull to customer wants. This assumes 
that customers could tell us what products to build and that this 
would act as the pull signal to product development to make 
them.? 

As was mentioned earlier, this is not the way the Lean 
Startup model works, because customers often don’t know what 
they want. Our goal in building products is to be able to run 
experiments that will help us learn how to build a sustainable 
business. Thus, the right way to think about the product devel- 
opment process in a Lean Startup is that it is responding to pull 
requests in the form of experiments that need to be run. 

As soon as we formulate a hypothesis that we want to test, 
the product development team should be engineered to design 
and run this experiment as quickly as possible, using the smallest 
batch size that will get the job done. Remember that although 
we write the feedback loop as Build-Measure-Learn because 
the activities happen in that order, our planning really works in 
the reverse order: we figure out what we need to learn and then 
work backwards to see what product will work as an experiment 
to get that learning. Thus, it is not the customer, but rather our 
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hypothesis about the customer, that pulls work from product de- 
velopment and other functions. Any other work is waste. 


Hypothesis Pull in Clean Tech 


To see this in action, let’s take a look at Berkeley-based startup 
Alphabet Energy. Any machine or process that generates power, 
whether it is a motor in a factory or a coal-burning power plant, 
generates heat as a by-product. Alphabet Energy has developed a 
product that can generate electricity from this waste heat, using 
a new kind of material called a thermoelectric. Alphabet En- 
ergy’ss thermoelectric material was developed over ten years by 
scientists at the Lawrence Berkeley National Laboratories. 

As with many clean technology products, there are huge chal- 
lenges in bringing a product like this to market. While working 
through its leap-of-faith assumptions, Alphabet figured out early 
that developing a solution for waste thermoelectricity required 
building a heat exchanger and a generic device to transfer heat 
from one medium to another as well as doing project-specific 
engineering. For instance, if Alphabet wanted to build a solu- 
tion for a utility such as Pacific Gas and Electric, the heat ex- 
_ changer would have to be configured, shaped, and installed to 
capture the heat from a power plant’s exhaust system. 

What makes Alphabet Energy unique is that the company 
made a savvy decision early on in the research process. Instead 
of using relatively rare elements as materials, they decided to 
base their research on silicon wafers, the same physical substance 
that computer central processing units (CPUs) are made from. 
As CEO Matthew Scullin explains, “Our thermoelectric is the 
only one that can use low-cost semiconductor infrastructure for 
manufacturing.” This has enabled Alphabet Energy to design 
and build its products in small batches. 

By contrast, most successful clean technology startups have 
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had to make substantial early investments. The solar panel pro- 
vider SunPower had to build in factories to manufacture its 
panels and partner with installers before becoming fully opera- 
tional. Similarly, BrightSource raised $291 million to build and 
operate large-scale solar plants without delivering a watt to a 
single customer. 

Instead of having to invest time and money in expensive 
fabrication facilities, Alphabet is able to take advantage of the 
massive existing infrastructure that produces silicon wafers for 
computer electronics. As a result, Alphabet can go from a prod- 
uct concept to holding a physical version in its hand in just six 
weeks from end to end. Alphabet’s challenge has been to find 
the combination of performance, price, and physical shape that 
is a match for early customers. Although its technology has rev- 
olutionary potential, early adopters will deploy it only if they 
can see a clear return on investment. 

It might seem that the most obvious market for Alphabet’s 
technology would be power plants, and indeed, that was the 
team’s initial hypothesis. Alphabet hypothesized that simple 
cycle gas turbines would be an ideal application; these turbines, 
which are similar to jet engines strapped to the ground, are used 
by power generators to provide energy for peak demand. Alpha- 
bet believed that attaching its semiconductors to those turbines 
would be simple and cheap. 

The company went about testing this hypothesis in small 
batches by building small-scale solutions for its customers as a 
way of learning. As with many initial ideas, their hypothesis was 
disproved quickly. Power companies have a low tolerance for 
risk, making them unlikely to become early adopters. Because it 
wasnt weighed down by a large-batch approach, Alphabet was 
ready to pivot after just three months of investigation. 

Alphabet has eliminated many other potential markets 
as well, leading to a series of customer segment pivots. The 
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company’s current efforts are focused on manufacturing firms, 
which have the ability to experiment with new technologies 
in separate parts of their factory; this allows early adopters to 
evaluate the real-world benefits before committing to a larger 
deployment. These early deployments are putting more of Al- 
phabet’s assumptions to the test. Unlike in the computer hard- 
ware business, customers are not willing to pay top dollar for 
maximum performance. This has required significant changes in 
Alphabet’s product, configuring it to achieve the lowest cost per 
watt possible. 

All this experimentation has cost the company a tiny fraction 
of what other energy startups have consumed. To date, Alphabet 
has raised approximately $1 million. Only time will tell if they 
will prevail, but thanks to the power of small batches, they will 
be able to discover the truth much faster."® 


000 


The Toyota Production System is probably the most advanced 
system of management in the world, but even more impressive 
is the fact that Toyota has built the most advanced learning or- 
ganization in history. It has demonstrated an ability to unleash 
the creativity of its employees, achieve consistent growth, and 
produce innovative new products relentlessly over the course of 
nearly a century." 

This is the kind of long-term success to which entrepreneurs 
should aspire. Although lean production techniques are power- 
ful, they are only a manifestation of a high-functioning organi- 
zation that is committed to achieving maximum performance 
by employing the right measures of progress over the long term. 
Process is only the foundation upon which a great company 
culture can develop. But without this foundation, efforts to 
encourage learning, creativity, and innovation will fall flat—as 
many disillusioned directors of HR can attest. 
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The Lean Startup works only if we are able to build an or- 
ganization as adaptable and fast as the challenges it faces. This 
requires tackling the human challenges inherent in this new way 
of working; that is the subject of the remainder of Part Three. 


THE STARTUP WAY 


PROCESS 
ACCOUNTABILITY 





I 
GROW 


recently had two startups seek my advice on the same day. As types 

of businesses, they could not have been more different. The 
first is developing a marketplace to help traders of collectibles 
connect with one another. These people are hard-core fans of 
movies, anime, or comics who strive to put together complete 
collections of toys and other promotional merchandise re- 
lated to the characters they love. The startup aspires to com- 
pete with online marketplaces such as eBay as well as physical 
marketplaces attached to conventions and other gatherings 
of fans. 

The second startup sells database software to enterprise cus- 
tomers. They have a next-generation database technology that 
can supplement or replace offerings from large companies such 
as Oracle, IBM, and SAP. Their customers are chief informa- 
tion officers (CIOs), IT managers, and engineers in some of the 
world’s largest organizations. These are long-lead-time sales that 
require salespeople, sales engineering, installation support, and 
maintenance contracts. 

You could be forgiven for thinking these two companies have 
absolutely nothing in common, yet both came to me with the 
exact same problem. Each one had early customers and prom- 
ising early revenue. They had validated and invalidated many 
hypotheses in their business models and were executing against 
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their product road maps successfully. Their customers had pro- 
vided a healthy mix of positive feedback and suggestions for 
improvements. Both companies had used their early success to 
raise money from outside investors. 

The problem was that neither company was growing. 

Both CEOs brought me identical-looking graphs showing that 
their early growth had flatlined. They could not understand why. 
_ They were acutely aware of the need to show progress to their em- 
ployees and investors and came to me because they wanted advice 
on how to jump-start their growth. Should they invest in more 
advertising or marketing programs? Should they focus on product 
quality or new features? Should they try to improve conversion 
rates or pricing? 

As it turns out, both companies share a deep similarity in the 
way their businesses grow—and therefore a similar confusion 
about what to do. Both are using the same engine of growth, the 
" topic of this chapter. 


WHERE DOES GROWTH COME FROM? 


The engine of growth is the mechanism that startups use to 
achieve sustainable growth. I use the word sustainable to ex- 
clude all one-time activities that generate a surge of customers 
but have no long-term impact, such as a single advertisement or 
a publicity stunt that might be used to jump-start growth but 
could not sustain that growth for the long term. 

Sustainable growth is characterized by one simple rule: 


New customers come from the actions of past customers. 


There are four primary ways past customers drive sustainable 
growth: 
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1. Word of mouth. Embedded in most products is a natural 
level of growth that is caused by satisfied customers’ enthusi- 
asm for the product. For example, when I bought my first TiVo 
DVR, I couldn't stop telling my friends and family about it. 
Pretty soon, my entire family was using one. 


2. As a side effect of product usage. Fashion or status, such 
as luxury goods products, drive awareness of themselves whenever 
they are used. When you see someone dressed in the latest clothes 
or driving a certain car, you may be influenced to buy that prod- 
uct. This is also true of so-called viral products such as Facebook 
and PayPal. When a customer sends money to a friend using Pay- 
Pal, the friend is exposed automatically to the PayPal product. 


3. Through funded advertising. Most businesses employ 
advertising to entice new customers to use their products. For 
this to be a source of sustainable growth, the advertising must 
be paid for out of revenue, not one-time sources such as invest- 
ment capital. As long as the cost of acquiring a new customer 
(the so-called marginal cost) is less than the revenue that cus- 
tomer generates (the marginal revenue), the excess (the marginal 
profit) can be used to acquire more customers. The more mar- 


ginal profit, the faster the growth. 


4. Through repeat purchase or use. Some products are de- 
signed to be purchased repeatedly either through a subscription 
plan (a cable company) or through voluntary repurchases (groceries 
or lightbulbs). By contrast, many products and services are inten- 
tionally designed as one-time events, such as wedding planning. 


These sources of sustainable growth power feedback loops that I 
have termed engines of growth. Each is like a combustion engine, 
turning over and over. The faster the loop turns, the faster the 
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company will grow. Each engine has an intrinsic set of metrics 
that determine how fast a company can grow when using it. 


THE THREE ENGINES OF GROWTH 


We saw in Part Two how important it is for startups to use 
the right kind of metrics—actionable metrics—to evaluate 
their progress. However, this leaves a large amount of variety 
in terms of which numbers one should measure. In fact, one 
of the most expensive forms of potential waste for a startup is 
spending time arguing about how to prioritize new develop- 
ment once it has a product on the market. At any time, the 
company could invest its energy in finding new customers, 
servicing existing customers better, improving overall qual- 
ity, or driving down costs. In my experience, the discussions 
about these kinds of priority decisions can consume a a sub- 
stantial fraction of the company’s time. 

Engines of growth are designed to give startups a relatively 
small set of metrics on which to focus their energies. As one of 
my mentors, the venture capital investor Shawn Carolan, put 
it, “Startups don't starve; they drown.” There are always a zil- 
lion new ideas about how to make the product better floating 
around, but the hard truth is that most of those ideas make a 
difference only at the margins. They are mere optimizations. 
Startups have to focus on the big experiments that lead to vali- 
dated learning. The engines of growth framework helps them 
stay focused on the metrics that matter. 


The Sticky Engine of Growth 
This brings us back to the two startups that kicked off this 


chapter. Both are using the exact same engine of growth despite 
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being in very different industries. Both products are designed 
to attract and retain customers for the long term. The underly- 
ing mechanism of that retention is different in the two cases. 
For the collectible company, the idea is to become the num- 
ber one shopping destination for fanatical collectors. These are 
people who are constantly hunting for the latest items and the 
best deals. If the company’s product works as designed, collec- 
tors who start using it will check constantly and repeatedly to 
see if new items are for sale as well as listing their own items for 
sale or trade. 

The startup database vendor relies on repeat usage for a very 
different reason. Database technology is used only as the founda- 
tion for a customer's own products, such as a website or a point 
of sale system. Once you build a product on top of a particular 
database technology, it is extremely difficult to switch. In the IT 
industry, such customers are said to be locked in to the vendor 
they choose. For such a product to grow, it has to offer such 
a compelling new capability that customers are willing to risk 
being tied to a proprietary vendor for a potentially long time. 

Thus, both businesses rely on having a high customer reten- 
tion rate. They have an expectation that once you start using 
their product, you will continue to do so. This is the same 
dynamic as a mobile telephone service provider: when a cus- 
tomer cancels his or her service, it generally means that he or 
she is extremely dissatisfied or is switching to a competitor's 
product. This is in contrast to, say, groceries on a store aisle. In 
the grocery retail business, customer tastes fluctuate, and if a 
customer buys a Pepsi this week instead of Coke, it’s not neces- 
sarily a big deal. 

Therefore, companies using the sticky engine of growth track 
their attrition rate or churn rate very carefully. The churn rate 
is defined as the fraction of customers in any period who fail to 
remain engaged with the company’s product. 
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The rules that govern the sticky engine of growth are pretty 
simple: if the rate of new customer acquisition exceeds the churn 
rate, the product will grow. The speed of growth is determined 
by what I call the rate of compounding, which is simply the nat- 
ural growth rate minus the churn rate. Like a bank account that 
earns compounding interest, having a high rate of compound- 
ing will lead to extremely rapid growth—without advertising, 
viral growth, or publicity stunts. : 

Unfortunately, both of these sticky startups were tracking 
their progress using generic indicators such as the total number 
_of customers. Even the actionable metrics they were using, such 
as the activation rate and revenue per customer, weren't very 
helpful because in the sticky engine of growth, these variables 
have little impact on growth. (In the sticky engine of growth, 
they are better suited to testing the value hypothesis that was 
discussed in Chapter 5.) 

After our meeting, one of the two startups took me up on my 
advice to model its customer behavior by using the sticky engine 
of growth as a template. The results were striking: a 61 percent 
retention rate and a 39 percent growth rate of new customers. 
In other words, its churn rate and new customer acquisition bal- 
anced each other almost perfectly, leading to a compounding 
growth rate of just 0.02 percent—almost zero. 

This is typical for companies in an engagement business 
that are struggling to find growth. An insider who worked at 
the dot-com-era company PointCast once showed me how that 
company suffered a similar dysfunction. When PointCast was 
struggling to grow, it was nonetheless incredibly successful in 
new customer acquisition—just like this sticky startup (39 per- 
cent every period). Unfortunately, this growth is being offset by 
an equivalent amount of churn. Once it is modeled this way, 
the good news should be apparent: there are plenty of new cus- 
tomers coming in the door. The way to find growth is to focus 
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on existing customers for the product even more engaging to 
them. For example, the company could focus on getting more 
and better listings. This would create an incentive for customers 
to check back often. Alternatively, the company could do some- 
thing more direct such as messaging them about limited-time 
sales or special offers. Either way, its focus needs to be on im- 
proving customer retention. This goes against the standard intu- 
ition in that if a company lacks growth, it should invest more in 
sales and marketing. This counterintuitive result is hard to infer 
from standard vanity mettics. 


The Viral Engine of Growth 


Online social networks and Tupperware are examples of prod- 
ucts for which customers do the lion’s share of the marketing. 
Awareness of the product spreads rapidly from person to per- 
son similarly to the way a virus becomes an epidemic. This is 
distinct from the simple word-of-mouth growth discussed 
above. Instead, products that exhibit viral growth depend on 
person-to-person transmission as a necessary consequence of 
normal product use. Customers are not intentionally acting as 
evangelists; they are not necessarily trying to spread the word 
about the product. Growth happens automatically as a side ef- 
fect of customers using the product. Viruses are not optional. 
For example, one of the most famous viral success stories is a 
company called Hotmail. In 1996, Sabeer Bhatia and Jack Smith 
launched a new web-based e-mail service that offered customers 
free accounts. At first, growth was sluggish; with only a small 
seed investment from the venture capital firm Draper Fisher 
Jurvetson, the Hotmail team could not afford an extensive mar- 
keting campaign. But everything changed when they made one 
small tweak to the product. They added to the bottom of every 
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single e-mail the message “P.S. Get your free e-mail at Hotmail” 
along with a clickable link. 

Within weeks, that small product change produced mas- 
sive results. Within six months, Bhatia and Smith had signed 
up more than 1 million new customers. Five weeks later, they 
hit the 2 million mark. Eighteen months after launching the 
service, with 12 million subscribers, they sold the company to 
Microsoft for $400 million.’ 

The same phenomenon is at work in Tupperware’s famous 
“house parties,” in which customers earn commissions by sell- 
ing the product to their friends and neighbors. Every sales pitch 
is an opportunity not only to sell Tupperware products but also 
to persuade other customers to become Tupperware representa- 
tives. Tupperware parties are still going strong decades after they 
started. Many other contemporary companies, such as Pam- 
pered Chef (owned by Warren Buffett’s Berkshire Hathaway), 
Southern Living, and Tastefully Simple, have adopted a similar 
model successfully. 

Like the other engines of growth, the viral engine is pow- 
ered by a feedback loop that can be quantified. It is called the 
viral loop, and its speed is determined by a single mathematical 
term called the viral coefficient. The higher this coefficient is, 
the faster the product will spread. The viral coefficient measures 
how many new customers will use a product as a consequence of 
each new customer who signs up. Put another way, how many 
friends will each customer bring with him or her? Since each 
friend is also a new customer, he or she has an opportunity to 
recruit yet more friends. 

For a product with a viral coefficient of 0.1, one in every ten 
customers will recruit one of his or her friends. This is not a 
sustainable loop. Imagine that one hundred customers sign up. 
They will cause ten friends to sign up. Those ten friends will 
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cause one additional person to sign up, but there the loop will 
fizzle out. 

By contrast, a viral loop with a coefficient that is greater than 
1.0 will grow exponentially, because each person who signs up 
will bring, on average, more than one other person with him 
or her. | 

To see these effects graphically, take a look at this chart: 
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Companies that rely on the viral engine of growth must 
focus on increasing the viral coefficient more than anything else, 
because even tiny changes in this number will cause dramatic 
changes in their future prospects. 

A consequence of this is that many viral products do not 
charge customers directly but rely on indirect sources of revenue 
such as advertising. This is the case because viral products can- 
not afford to have any friction impede the process of signing 
customers up and recruiting their friends. This can make testing 
the value hypothesis for viral products especially challenging. 

The true test of the value hypothesis is always a voluntary 
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exchange of value between customers and the startup that serves 
them. A lot of confusion stems from the fact that this exchange 
can be monetary, as in the case of Tupperware, or nonmonetary, 
as in the case of Facebook. In the viral engine of growth, mon- 
etary exchange does not drive new growth; it is useful only as 
an indicator that customers value the product enough to pay 
for it. If Facebook or Hotmail had started charging customers 
in their early days, it would have been foolish, as it would have 
impeded their ability to grow. However, it is not true that cus- 
tomers do not give these companies something of value: by in- 
vesting their time and attention in the product, they make the 
product valuable to advertisers. Companies that sell advertising 
actually serve two different groups of customers—consumers 
and advertisers—and exchange a different currency of value 
with each.’ 

This is markedly different from companies that actively use 
money to fuel their expansion, such as a retail chain that can 
grow as fast as it can fund the opening of new stores at suit- 
able locations. These companies are using a different engine of 


growth altogether. 


The Paid Engine of Growth 


Imagine another pair of businesses. The first makes $1 on each 
customer it signs up; the second makes $100,000 from each 
customer it signs up. To predict which company will grow faster, 
you need to know only one additional thing: how much it costs 
to sign up a new customer. 

Imagine that the first company uses Google AdWords to find 
new customers online and pays an average of 80 cents each time 
a new customer joins. The second company sells heavy goods to 
large companies. Each sale requires a significant time investment 
from a salesperson and on-site sales engineering to help install 
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the product; these hard costs total up to $80,000 per new cus- 
tomer. Both companies will grow at the exact same rate. Each 
has the same proportion of revenue (20 percent) available to 
reinvest in new customer acquisition. If either company wants 
to increase its rate of growth, it can do so in one of two ways: 
increase the revenue from each customer or drive down the cost 
of acquiring a new customer. 

That's the paid engine of growth at work. 

In relating the IMVU story in Chapter 3, I talked about how 
we made a major early mistake in setting up the IMVU strategy. 
We ultimately wound up having to make an engine of growth 
pivot. We originally thought that our IM add-on strategy would 
allow the product to grow virally. Unfortunately, customers re- 
fused to go along with our brilliant strategy. 

Our basic misconception was a belief that customers would 
be willing to use IMVU as an add-on to existing instant messag- 
ing networks. We believed that the product would spread virally 
through those networks, passed from customer to customer. 
The problem with that theory is that some kinds of products are 
not compatible with viral growth. 

IMVU’s customers didn’t want to use the product with their 
existing friends. They wanted to use it to make new friends. Un- 
fortunately, that meant they did not have a strong incentive to 
bring new customers to the product; they viewed that as our 
job. Fortunately, IMVU was able to grow by using paid adver- 
tising because our customers were willing to pay more for our 
product than it cost us to reach them via advertising. 

Like the other engines, the paid engine of growth is pow- 
ered by a feedback loop. Each customer pays a certain amount 
of money for the product over his or her “lifetime” as a cus- 
tomer. Once variable costs are deducted, this usually is called 
the customer lifetime value (LTV). This revenue can be invested 
in growth by buying advertising. 
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Suppose an advertisement costs $100 and causes fifty new 
customers to sign up for the service. This ad has a cost per ac- 
quisition (CPA) of $2.00. In this example, if the product has an 
LTV that is greater than $2, the product will grow. The margin 
between the LTV and the CPA determines how fast the paid 
engine of growth will turn (this is called the marginal profit). 
Conversely, if the CPA remains at $2.00 but the LTV falls below 
$2.00, the company’s growth will slow. It may make up the dif- 
ference with one-time tactics such as using invested capital or 
publicity stunts, but those tactics are not sustainable. ‘This was 
the fate of many failed companies, including notable dot-com 
flameouts that erroneously believed that they could lose money 
on each customer but, as the old joke goes, make it up in volume. 

Although I have explained the paid engine of growth in 
terms of advertising, it is far broader than that. Startups that 
employ an outbound sales force are also using this engine, as are 
retail companies that rely on foot traffic. All these costs should 
be factored into the cost per acquisition. 

For example, one startup I worked with built collaboration 
tools for teams and groups. It went through a radical pivot, 
switching from a tool that was used primarily by hobbyists and 
small clubs to one that was sold primarily to enterprises, non- 
governmental organizations (NGOs), and other extremely large 
organizations. However, they made that customer segment pivot 
without changing their engine of growth. Previously, they had 
done customer acquisition online, using web-based direct mar- 
keting techniques. I remember one early situation in which the 
company fielded a call from a major NGO that wanted to buy its 
product and roll it out across many divisions. The startup had an 
“unlimited” pricing plan, its most expensive, that cost only a few 
hundred dollars per month. The NGO literally could not make 
the purchase because it had no process in place for buying some- 
thing so inexpensive. Additionally, the NGO needed substantial 
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help in managing the rollout, educating its staff on the new tool, 
and tracking the impact of the change; those were all services the 
company was ill equipped to offer. Changing customer segments 
required them to switch to hiring a sizable outbound sales staff 
that spent time attending conferences, educating executives, and 
authoring white papers. Those much higher costs came with a 
corresponding reward: the company switched from making only 
a few dollars per customer to making tens and then hundreds of 
thousands of dollars per much larger customer. Their new engine 
of growth led to sustained success. 

Most sources of customer acquisition are subject to competi- 
tion. For example, prime retail storefronts have more foot traf- 
fic and are therefore more valuable. Similarly, advertising that is 
targeted to more affluent customers generally costs more than 
advertising that reaches the general public. What determines 
these prices is the average value earned in aggregate by the com- 
panies that are in competition for any given customer's atten- 
tion. Wealthy consumers cost more to reach because they tend 
to become more profitable customers. 

Over time, any source of customer acquisition will tend to 
have its CPA bid up by this competition. If everyone in an in- 
dustry makes the same amount of money on each sale, they all 
will wind up paying most of their marginal profit to the source 
of acquisition. Thus, the ability to grow in the long term by 
using the paid engine requires a differentiated ability to mon- 
etize a certain set of customers. 

IMVU 1s a case in point. Our customers were not consid- 
ered very lucrative by other online services: they included a lot 
of teenagers, low-income adults, and international customers. 
Other services tended to assume those people would not pay 
for anything online. At IMVU, we developed techniques for 
collecting online payments from customers who did not have a 
credit card, such as allowing them to bill to their mobile phones 
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or send us cash in the mail. Therefore, we could afford to pay 
more to acquire those customers than our competitors could. 


A Technical Caveat 


Technically, more than one engine of growth can operate in a 
business at a time. For example, there are products that have ex- 
tremely fast viral growth as well as extremely low customer churn 
rates. Also, there is no reason why a product cannot have both 
high margins and high retention. However, in my experience, 
successful startups usually focus on just one engine of growth, 
specializing in everything that is required to make it work. 
Companies that attempt to build a dashboard that includes all 
three engines tend to cause a lot of confusion because the op- 
erations expertise required to model all these effects simultane- 
ously is quite complicated. Therefore, I strongly recommend 
that startups focus on one engine at a time. Most entrepreneurs 
already have a strong leap-of-faith hypothesis about which en- 
gine is most likely to work. If they do not, time spent out of 
the building with customers will quickly suggest one that seems 
profitable. Only after pursuing one engine thoroughly should a 


startup consider a pivot to one of the others. 


ENGINES OF GROWTH DETERMINE PRODUCT /MARKET FIT 


Marc Andreessen, the legendary entrepreneur and investor and 
one of the fathers of the World Wide Web, coined the term prod- 
uct/market fit to describe the moment when a startup finally finds 
a widespread set of customers that resonate with its product: 


In a great market—a market with lots of real potential 
customers—the market pulls product out of the startup. 
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This is the story of search keyword advertising, Internet 
auctions, and TCP/IP routers. Conversely, in a terrible 
market, you can have the best product in the world and 
an absolutely killer team, and it doesn’t matter—you're 
going to fail.” 


When you see a startup that has found a fit with a large mar- 
ket, it’s exhilarating. It leaves no room for doubt. It is Ford’s 
Model T flying out of the factory as fast as it could be made, 
Facebook sweeping college campuses practically overnight, or 
Lotus taking the business world by storm, selling $54 million 
worth of Lotus 1-2-3 in its first year of operation. 

Startups occasionally ask me to help them evaluate whether 
they have achieved product/market fit. It’s easy to answer: if you 
are asking, you're not there yet. Unfortunately, this doesn’t help 
companies figure out /ow to get closer to product/market fit. 
How can you tell if you are on the verge of success or hopelessly 
far away? 

Although I don't think Andreessen intended this as part of 
his definition, to many entrepreneurs it implies that a pivot is 
a failure event—“our startup has failed to achieve product/mar- 
ket fit.” It also implies the inverse—that once our product has 
achieved product/market fit, we won't have to pivot anymore. 
Both assumptions are wrong. 

I believe the concept of the engine of growth can put the idea 
of product/market fit on a more rigorous footing. Since each en- 
gine of growth can be defined quantitatively, each has a unique 
set of metrics that can be used to evaluate whether a startup is on 
the verge of achieving product/market fit. A startup with a viral 
coefficient of 0.9 or more is on the verge of success. Even better, 
the metrics for each engine of growth work in tandem with the 
innovation accounting model discussed in Chapter 7 to give di- 
rection to a startups product development efforts. For example, 
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if a startup is attempting to use the viral engine of growth, it can 
focus its development efforts on things that might affect cus- 
tomer behavior—on the viral loop—and safely ignore those that 
do not. Such a startup does not need to specialize in marketing, 
advertising, or sales functions. Conversely, a company using the 
paid engine needs to develop those marketing and sales func- 
tions urgently. 

A startup can evaluate whether it is getting closer to product/ 
market fit as it tunes its engine by evaluating each trip through 
the Build-Measure-Learn feedback loop using innovation ac- 
counting. What really matters is not the raw numbers or vanity 
metrics but the direction and degree of progress. 

For example, imagine two startups that are working diligently 
to tune the sticky engine of growth. One has a compounding 
rate of growth of 5 percent, and the other 10 percent. Which 
company is the better bet? On the surface, it may seem that the 
larger rate of growth is better, but what if each company’s in- 
novation accounting dashboard looks like the following chart? 


Even with no insight into these two companies gross num- 
bers, we can tell that company A is making real progress whereas 
company B is stuck in the mud. This is true even though com- 
pany B is growing faster than company A right now. 
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WHEN ENGINES RUN OUT 


Getting a startups engine of growth up and running is hard 
enough, but the truth is that every engine of growth eventually 
runs out of gas. Every engine is tied to a given set of customers 
and their related habits, preferences, advertising channels, and 
interconnections. At some point, that set of customers will be 
exhausted. This may take a long time or a short time, depending 
on one’s industry and timing. 

Chapter 6 emphasized the importance of building the mini- 
mum viable product in such a way that it contains no additional 
features beyond what is required by early adopters. Following 
that strategy successfully will unlock an engine of growth that 
can reach that target audience. However, making the transition 
to mainstream customers will require tremendous additional 
work.* Once we have a product that is growing among early 
adopters, we could in theory stop work in product development 
entirely. The product would continue to grow until it reached 
the limits. of that early market. Then growth would level off or 
even stop completely. The challenge comes from the fact that 
this slowdown might take months or even years to take place. 
Recall from Chapter 8 that IMVU failed this test—at first—for 
precisely this reason. 

Some unfortunate companies wind up following this strat- 
egy inadvertently. Because they are using vanity metrics and tra- 
ditional accounting, they think they are making progress when 
they see their numbers growing. They falsely believe they are 
making their product better when in fact they are having no 
impact on customer behavior. The growth is all coming from an 
engine of growth that is working—running efficiently to bring 
in new customers—not from improvements driven by product 
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development. Thus, when the owe suddenly slows, it pro- 
vokes a crisis. 

This is the same problem that established companies experi- 
ence. Their past successes were built on a finely tuned engine of 
growth. If that engine runs its course and growth slows or stops, 
there can be a crisis if the company does not have new start- 
ups incubating within its ranks that can provide new sources of 
growth. 

Companies of any size can suffer from this perpetual afflic- 
tion. They need to manage a portfolio of activities, simultane- 
ously tuning their engine of growth and developing new sources 
of growth for when that engine inevitably runs its course. How 
to do this is the subject of Chapter 12. However, before we can 
manage that portfolio, we need an organizational structure, cul- 
ture, and discipline that can handle these rapid and often unex- 
pected changes. I call this an adaptive organization, and it is the 
subject of Chapter 11. 


I 


ADAPT 


hen | was the CIO of IMVU, I thought I was doing a good 


job most of the time. I had built an agile engineering or- 
ganization, and we were successfully experimenting with the 
techniques that would come to be known as the Lean Startup. 
However, on a couple of occasions I suddenly realized that I was 
failing at my job. For an achievement-oriented person, that is 
incredibly disarming. Worst of all, you don’t get a memo. If you 


did, it would read something like this: 


Dear Eric, 

Congratulations! The job you used to do at this 
company is no longer available. However, you have been 
transferred to a new job in the company. Actually, it’s not 
the same company anymore, even though it has the same 
name and many of the same people. And although the 
job has the same title, too, and you used to be good at 
your old job, you're already failing at the new one. This 
transfer is effective as of six months ago, so this is to 
alert you that you've already been failing at it for quite 
some time. 


Best of luck! 
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Every time this happened to me, | struggled to figure out 
what to do. I knew that as the company grew, we would need 
additional processes and systems designed to coordinate the 
company’s operations at each larger size. And yet I had also seen 
many startups become ossified and bureaucratic out of a mis- 
placed desire to become “professional.” 

Having no system at all was not an option for IMVU and is 
not an option for you. There are so many ways for a startup to fail. 
I’ve lived through the overarchitecture failure, in which attempt- 
ing to prevent all the various kinds of problems that could occur 
wound up delaying the company from putting out any product. 
I’ve seen companies fail the other way from the so-called Friend- 
ster effect, suffering a high-profile technical failure just when 
customer adoption is going wild. As a department executive, this 
outcome is worst of all, because the failure is both high-profile 
and attributable to a single function or department—yours. Not 
only will the company fail, it will be your fault. 

Most of the advice I’ve heard on this topic has suggested a 
kind of split-the-difference approach (as in, “engage in a little 
planning, but not too much”). The problem with this willy-nilly 
approach is that it’s hard to give any rationale for why we should 
anticipate one particular problem but ignore another. It can feel 
like the boss is being capricious or arbitrary, and that feeds the 
common feeling that management's decisions conceal an ulte- 
rior motive. 

For those being managed this way, their incentives are clear. 
If the boss tends to split the difference, the best way to influ- 
ence the boss and get what you want is to take the most extreme 
position possible. For example, if one group is advocating for an 
extremely lengthy release cycle, say, an annual new product in- 
troduction, you might choose to argue for an equally extremely 
short release cycle (perhaps weekly or even daily), knowing that 
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the two opinions will be averaged out. Then, when the differ- 
ence is split, youre likely to get an outcome closer to what you 
actually wanted in the first place. Unfortunately, this kind of 
arms race escalates. Rivals in another camp are likely to do the 
same thing. Over time, everyone will take the most polarized 
positions possible, which makes splitting the difference ever 
more difficult and ever less successful. Managers have to take 
responsibility for knowingly or inadvertently creating such in- 
centives. Although it was not their intention to reward extreme 
polarization, that's exactly what they are doing. Getting out of 
this trap requires a significant shift in thinking. 


BUILDING AN ADAPTIVE ORGANIZATION 


Should a startup invest in a training program for new employ- 
ees? If you had asked me a few years ago, I would have laughed 
and said, “Absolutely not. Training programs are for big compa- 
nies that can afford them.” Yet at IMVU we wound up building 
a training program that was so good, new hires were produc- 
tive on their first day of employment. Within just a few weeks, 
those employees were contributing at a high level. It required 
a huge effort to standardize our work processes and prepare a 
curriculum of the concepts that new employees should learn. 
Every new engineer would be assigned a mentor, who would 
help the new employee work through a curriculum of systems, 
concepts, and techniques he or she would need to become pro- 
ductive at IMVU. The performance of the mentor and mentee 
were linked, so the mentors took this education seriously. 

What is interesting, looking back at this example, is that 
we never stopped work and decided that we needed to build a 
great training program. Instead, the training program evolved 
organically out of a methodical approach to evolving our own 
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process. This process of orientation was subject to constant ex- 
perimentation and revision so that it grew more effective—and 
less burdensome—over time. 

I call this building an adaptive organization, one that automat- 
ically adjusts its process and performance to current conditions. 


Can You Go Too Fast? 


So far this book has emphasized the importance of speed. Start- 
ups are in a life-or-death struggle to learn how to build a sustain- 
able business before they run out of resources and die. However, 
focusing on speed alone would be destructive. To work, startups 
require built-in speed regulators that help teams find their opti- 
mal pace of work. 

We saw an example of speed regulation in Chapter 9 with 
the use of the andon cord in systems such as continuous deploy- 
ment. It is epitomized in the paradoxical Toyota proverb, “Stop 
production so that production never has to stop.” The key to the 
andon cord is that it brings work to a stop as soon as an uncor- 
rectable quality problem surfaces—which forces it to be investi- 
gated. This is one of the most important discoveries of the lean 
manufacturing movement: you cannot trade quality for time. If 
you are causing (or missing) quality problems now, the resulting 
defects will slow you down later. Defects cause a lot of rework, 
low morale, and customer complaints, all of which slow progress 
and eat away at valuable resources. | 

So far I have used the language of physical products to de- 
scribe these problems, but that is simply a matter of conve- 
nience. Service businesses have the same challenges. Just ask 
any manager of a training, staffing, or hospitality firm to show 
you the playbook that specifies how employees are supposed to 
deliver the service under various conditions. What might have 
started out as a simple guide tends to grow inexorably over time. 
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Pretty soon, orientation is incredibly complex and employees 
have invested a lot of time and energy in learning the rules. 
Now consider an entrepreneurial manager in that kind of com- 
pany trying to experiment with new rules or procedures. The 
higher-quality the existing playbook is, the easier it will be for it 
to evolve over time. By contrast, a low-quality playbook will be 
filled with contradictory or ambiguous rules that cause confu- 
sion when anything is changed. 

When I teach the Lean Startup approach to entrepreneurs 
with an engineering background, this is one of the hardest con- 
cepts to grasp. On the one hand, the logic of validated learn- 
ing and the minimum viable product says that we should get a 
product into customers’ hands as soon as possible and that any 
extra work we do beyond what is required to learn from cus- 
tomers is waste. On the other hand, the Build-Measure-Learn 
feedback loop is a continuous process. We don’t stop after one 
minimum viable product but use what we have learned to get to 
work immediately on the next iteration. 

Therefore, shortcuts taken in product quality, isan or in- 
frastructure today may wind up slowing a company down to- 
morrow. You can see this paradox in action at IMVU. Chapter 
3 recounted how we wound up shipping a product to customers 
that was full of bugs, missing features, and bad design. The cus- 
tomers wouldn't even try that product, and so most of that work 
had to be thrown away. It’s a good thing we didn’t waste a lot of 
time fixing those bugs and cleaning up that early version. 

However, as our learning allowed us to build products that 
customers did want, we faced slowdowns. Having a low-quality 
product can inhibit learning when the defects prevent custom- 
ers from experiencing (and giving feedback on) the product's 
benefits. In IMVU’s case, as we offered the product to more 
mainstream customers, they were much less forgiving than early 
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adopters had been. Similarly, the more features we added to the 
product, the harder it became to add even more because of the 
risk that a new feature would interfere with an existing feature. 
The same dynamics happen in a service business, since any new 
rules may conflict with existing rules, and the more rules, the 
more possibilities for conflict. 

IMVU used the techniques of this chapter to achieve scale 


and quality in a just-in-time fashion. 


THE WISDOM OF THE FIVE WHYS 


To accelerate, Lean Startups need a process that provides a natural 
feedback loop. When you're going too fast, you cause more prob- 
lems. Adaptive processes force you to slow down and invest in 
preventing the kinds of problems that ate currently wasting time. 
As those preventive efforts pay off, you naturally speed up again. 
Let's return to the question of having a training program for 
new employees. Without a program, new employees will make 
mistakes while in their learning curve that will require assistance 
and intervention from other team members, slowing everyone 
down. How do you decide if the investment in training is worth 
the benefit of speed due to reduced interruptions? Figuring this 
out from a top-down perspective is challenging, because it re- 
quires estimating two completely unknown quantities: how 
much it will cost to build an unknown program against an un- 
known benefit you might reap. Even worse, the traditional way 
to make these kinds of decisions is decidedly large-batch think- 
ing. A company either has an elaborate training program or it 
does not. Until they can justify the return on investment from 
building a full program, most companies generally do nothing. 
The alternative is to use a system called the Five Whys to 
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make incremental investments and evolve a startup’s processes 
gradually. The core idea of Five Whys is to tie investments di- 
rectly to the prevention of the most problematic symptoms. The 
system takes its name from the investigative method of asking 
the question “Why?” five times to understand what has hap- 
pened (the root cause). If you've ever had to answer a precocious 
child who wants to know “Why is the sky blue?” and keeps 
asking “Why?” after each answer, youre familiar with it. This 
technique was developed as a systematic problem-solving tool 
by Taiichi Ohno, the father of the Toyota Production System. 
I have adapted it for use in the Lean Startup model with a few 
changes designed specifically for startups. 

At the root of every seemingly technical problem is a human 
problem. Five Whys provides an opportunity to discover what 
that human problem might be. Taiichi Ohno gives the follow- 
ing example: 


When confronted with a problem, have you ever stopped 
and asked why five times? It is difficult to do even though 
it sounds easy. For example, suppose a machine stopped 
functioning: 


1. Why did the machine stop? (There was an overload and 
the fuse blew.) 

2. Why was there an overload? (The bearing was not suf- 
ficiently lubricated.) | 

3. Why was it not lubricated sufficiently? (The lubrication 
pump was not pumping sufficiently.) 

4, Why was it not pumping sufficiently? (The shaft of the 
pump was worn and rattling.) 

5. Why was the shaft worn out? (There was no strainer at- 
tached and metal scrap got in.) 
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Repeating “why” five times, like this, can help uncover 
the root problem and correct it. If this procedure were 
not carried through, one might simply replace the fuse 
or the pump shaft. In that case, the problem would recur 
within a few months. The Toyota production system has 
been built on the practice and evolution of this scientific 
approach. By asking and answering “why” five times, we 
can get to the real cause of the problem, which is often 
hidden behind more obvious symptoms. ! 


Note that even in Ohno’ relatively simple example the root 
cause moves away from a technical fault (a blown fuse) and 
toward a human error (someone forgot to attach a strainer). 
This is completely typical of most problems that startups face 
no matter what industry they are in. Going back to our service 
business example, most problems that at first appear to be in- 
dividual mistakes can be traced back to problems in training or 
the original playbook for how the service is to be delivered. 

Let me demonstrate how using the Five Whys allowed us to 
build the employee training system that was mentioned earlier. 
Imagine that at IMVU we suddenly start receiving complaints 
from customers about a new version of the product that we have 
just released. 


1. A new release disabled a feature for customers. Why? Be- 
cause a particular server failed. 

2. Why did the server fail? Because an obscure subsystem 
was used in the wrong way. 

3. Why was it used in the wrong way? The engineer who 
used it didn’t know how to use it properly. 

4, Why didn’t he know? Because he was never trained. 

5. Why wasn't he trained? Because his manager doesn’t be- 
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lieve in training new engineers because he and his team 
are “too busy.” 


What began as a purely technical fault is revealed quickly to 
be a very human managerial issue. 


Make a Proportional Investment 


Here’s how to use Five Whys analysis to build an adaptive orga- 
nization: consistently make a proportional investment at each of 
the five levels of the hierarchy. In other words, the investment 
should be smaller when the symptom is minor and larger when 
the symptom is more painful. We don't make large investments 
in prevention unless we're coping with large problems. 

In the example above, the answer is to fix the server, change 
the subsystem to make it less error-prone, educate the engineer, 
and, yes, have a conversation with the engineer's manager. 

This latter piece, the conversation with the manager, is al- 
ways hard, especially in a startup. When I was a startup man- 
ager, if you told me I needed to invest in training my people, 
I would have told you it was a waste of time. There were al- 
ways too many other things to do. I'd probably have said some- 
thing sarcastic like “Sure, I'd be happy to do that—if you can — 
spare my time for the eight weeks it'll take to set up.” That's 
manager-speak for “No way in hell.” 

That’s why the proportional investment approach is so im- 
portant. If the outage is a minor glitch, it’s essential that we 
make only a minor investment in fixing it. Let’s do the first hour 
of the eight-week plan. That may not sound like much, but it’s 
a start. If the problem recurs, asking the Five Whys will require 
that we continue to make progress on it. If the problem does not 
occur again, an hour isn’t a big loss. 

I used the example of engineering training because that was 
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something I was reluctant to invest in at IMVU. At the outset 
of our venture, I thought we needed to focus all of our energies 
on building and marketing our product. Yet once we entered a 
period of rapid hiring, repeated Five Whys sessions revealed that 
problems caused by lack of training were slowing down prod- 
uct development. At no point did we drop everything to focus 
solely on training. Instead, we made incremental improvements 
to the process constantly, each time reaping incremental ben- 
efits. Over time, those changes compounded, freeing up time 
and energy that previously had been lost to firefighting and cri- 
sis management. 


Automatic Speed Regulator 


The Five Whys approach acts as a natural speed regulator. The 
more problems you have, the more you invest in solutions to 
those problems. As the investments in infrastructure or process 
pay off, the severity and number of crises are reduced and the 
team speeds up again. With startups in particular, there is a dan- 
ger that teams will work too fast, trading quality for time in a 
way that causes sloppy mistakes. Five Whys prevents that, allow- 
ing teams to find their optimal pace. 

The Five Whys ties the rate of progress to learning, not just 
execution. Startup teams should go through the Five Whys 
whenever they encounter any kind of failure, including tech- 
nical faults, failures to achieve business results, or unexpected 
changes in customer behavior. 

Five Whys is a powerful organizational technique. Some of 
the engineers I have trained to use it believe that you can derive 
all the other Lean Startup techniques from the Five Whys. Cou- 
pled with working in small batches, it provides the foundation a 
company needs to respond quickly to problems as they appear, 
without overinvesting or overengineering. 
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THE CURSE OF THE FIVE BLAMES 


When teams first adopt Five Whys as a problem-solving tool, 
they encounter some common pitfalls. We need systems like Five 
Whys to overcome our psychological limitations because we tend 
to overreact to what's happening in the moment. We also tend to 
get frustrated if things happen that we did not anticipate. 
_ When the Five Whys approach goes awry, I call it the Five 
Blames. Instead of asking why repeatedly in an attempt to un- 
derstand what went wrong, frustrated teammates start pointing 
fingers at each other, trying to decide who is at fault. Instead 
of using the Five Whys to find and fix problems, managers 
and employees can fall into the trap of using the Five Blames 
as a means for venting their frustrations and calling out col- 
leagues for systemic failures. Although it’s human nature to as- 
sume that when we see a mistake, it’s due to defects in someone 
else’s department, knowledge, or character, the goal of the Five 
Whys is to help us see the objective truth that chronic prob- 
lems are caused by bad process, not bad people, and remedy 
them accordingly. 

I recommend several tactics for escaping the Five Blames. 
The first is to make sure that everyone affected by the problem 
is in the room during the analysis of the root cause. The meeting 
should include anyone who discovered or diagnosed the prob- 
lem, including customer service representatives who fielded the 
calls, if possible. It should include anyone who tried to fix the 
symptom as well as anyone who worked on the subsystems or 
features involved. If the problem was escalated to senior man- 
agement, the decision makers who were involved in the escala- 
tion should be present as well. 

This may make for a crowded room, but it’s essential. In my 
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experience, whoever is left out of the discussion ends up being 
the target for blame. This is just as damaging whether the scape- 
goat is a junior employee or the CEO. When it’s a junior em- 
ployee, it’s all too easy to believe that that person is replaceable. 
If the CEO is not present, it’s all too easy to assume that his or 
her behavior is unchangeable. Neither presumption is usually 
correct. 

When blame inevitably arises, the most senior people in the 
room should repeat this mantra: if a mistake happens, shame 
on us for making it so easy to make that mistake. In a Five 
Whys analysis, we want to have a systems-level view as much as 
possible. 

Here's a situation in which this mantra came in handy. 
Because of the training process we had developed at IMVU 
through the Five Whys, we routinely asked new engineers to 
make a change to the production environment on their first day. 
For engineers trained in traditional development methods, this 
was often frightening. They would ask, “What will happen to 
me if I accidentally disrupt or stop the production process?” 
In their previous jobs, that was a mistake that could get them 
fired. At IMVU we told new hires, “If our production process 
is so fragile that you can break it on your very first day of work, 
shame on us for making it so easy to do so.” If they did manage 
to break it, we immediately would have them lead the effort to 
fix the problem as well as the effort to prevent the next person 
from repeating their mistake. | 

For new hires who came from companies with a very differ- 
ent culture, this was often a stressful initiation, but everyone 
came through it with a visceral understanding of our values. 
Bit by bit, system by system, those small investments added up 
to a robust product development process that allowed all our 
employees to work more creatively, with greatly reduced fear. 
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Getting Started 


Here are a few tips on how to get started with the Five Whys 
that are based on my experience introducing this technique at 
many other companies. 

For the Five Whys to work properly, there are rules that must 
be followed. For example, the Five Whys requires an environ- 
ment of mutual trust and empowerment. In situations in which 
this is lacking, the complexity of Five Whys can be overwhelm- 
ing. In such situations, I’ve often used a simplified version that 
still allows teams to focus on analyzing root causes while de- 
veloping the muscles they'll need later to tackle the full-blown 
method. 


I ask teams to adopt these simple rules: 


1. Be tolerant of all mistakes the first time. 
2. Never allow the same mistake to be made twice. 


The first rule encourages people to get used to being com- 
passionate about mistakes, especially the mistakes of others. Re- 
member, most mistakes are caused by flawed systems, not bad 
people. The second rule gets the team started making propor- 
tional investments in prevention. 

This simplified system works well. In fact, we used it at 
IMVU in the days before I discovered the Five Whys and the 
Toyota Production System. However, such a simplified system 
does not work effectively over the long term, as I found out first- 
hand. In fact, that was one of the things that drove me to first 
learn about lean production. 

The strength and weakness of the simplified system is that 
it invites questions such as What counts as the same problem? 


What kinds of mistakes should we focus on? and Should we fix 
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this individual problem or try to prevent a whole category of 
related problems? For a team that is just getting started, these 
questions are thought-provoking and can lay the groundwork 
for more elaborate methods to come. Ultimately, though, they 


do need answering. They need a complete adaptive process such 
as the Five Whys. 


Facing Unpleasant Truths 


You will need to be prepared for the fact that Five Whys is going 
to turn up unpleasant facts about your organization, especially 
at the beginning. It is going to call for investments in preven- 
tion that come at the expense of time and money that could be 
invested in new products or features. Under pressure, teams may 
feel that they don’t have time to waste on analyzing root causes 
even though it would give them more time in the long term. 
The process sometimes will devolve into the Five Blames. At 
all these junctures, it is essential that someone with sufficient 
authority be present to insist that the process be followed, that 
its recommendations be implemented, and to act as a referee 
if disagreements flare up. Building an adaptive organization, in 
other words, requires executive leadership to sponsor and sup- 
port the process. 

Often, individual contributors at startups come to my work- 
shops, eager to get started with the Five Whys. I caution against 
attempting to do that if they do not have the buy-in of the 
manager or team leader. Proceed cautiously if you find yourself 
in this situation. It may not be possible to get the entire team 
together for a true Five Whys inquiry, but you can always fol- 
low the simple two-rule version in your own work. Whenever 
something goes wrong, ask yourself: How could I prevent my- 
self from being in this situation ever again? 


238 = THE LEAN STARTUP 


Start Small, Be Specific 


Once you are ready to begin, I recommend starting with a nar- 
rowly targeted class of symptoms. For example, the first time 
I used the Five Whys successfully, I used it to diagnose prob- 
lems with one of our internal testing tools that did not affect 
customers directly. It may be tempting to start with something 
large and important because that is where most of the time is 
being wasted as a result of a flawed process, but it is also where 
the pressure will be greatest. When the stakes are high, the Five 
Whys can devolve into the Five Blames quickly. It’s better to 
give the team a chance to learn how to do the process first and 
then expand into higher-stakes areas later. 

The more specific the symptoms are, the easier it will be for 
everyone to recognize when it's time to schedule a Five Whys 
meeting. Say you want to use the Five Whys to address billing 
complaints from customers. In that case, pick a date after which 
all billing complaints will trigger a Five Whys meeting automati- 
cally. Note that this requires that there be a small enough volume 
of complaints that having this meeting every time one comes in is 
practical. If there are already too many complaints, pick a subset 
on which you want to focus. Make sure that the rule that deter- 
mines which kinds of complaints trigger a Five Whys meeting is 
simple and ironclad. For example, you might decide that every 
complaint involving a credit card transaction will be investigated. 
That’s an easy rule to follow. Don’t pick a rule that is ambiguous. 

At first, the temptation may be to make radical and deep 
changes to every billing system and process. Don’. Instead, keep 
the meetings short and pick relatively simple changes at each of 
the five levels of the inquiry. Over time, as the team gets more 
comfortable with the process, you can expand it to include more 
and more types of billing complaints and then to other kinds of 
problems. 
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Appoint a Five Whys Master 


To facilitate learning, I have found it helpful to appoint a Five 
Whys master for each area in which the method is being used. 
This individual is tasked with being the moderator for each Five 
Whys meeting, making decisions about which prevention steps 
to take, and assigning the follow-up work from that meeting. 
The master must be senior enough to have the authority to en- 
sure that those assignments get done but should not be so senior 
that he or she will not be able to be present at the meetings be- 
cause of conflicting responsibilities. The Five Whys master is the 
point person in terms of accountability; he or she is the primary 
change agent. People in this position can assess how well the 
meetings are going and whether the prevention investments that 
are being made are paying off. 


THE FIVE WHYS IN ACTION 


IGN Entertainment, a division of News Corporation, is an on- 
line video games media company with the biggest audience of 
video game players in the world. More than 45 million gamers 
frequent its portfolio of media properties. IGN was founded in 
the late 1990s, and News Corporation acquired it in 2005. IGN 
has grown to employ several hundred people, including almost 
a hundred engineers. 

Recently, I had the opportunity to speak to the product de- 
velopment team at IGN. They had been successful in recent 
years, but like all the established companies we've seen through- 
out this book, they were looking to accelerate new product de- 
velopment and find ways to be more innovative. They brought 
together their engineering, product, and design teams to talk 
through ways they could apply the Lean Startup model. 
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This change initiative had the support of IGN’s senior man- 
agement, including the CEO, the head of product develop- 
ment, the vice president of engineering, the publisher, and the 
head of product. Their previous efforts at Five Whys had not 
gone smoothly.. They had attempted to tackle a laundry list of 
problem areas nominated by the product team. The issues var- 
ied from discrepancies in web analytics to partner data feeds that 
were not working. Their first Five Whys meeting took an hour, 
and although they came up with some interesting takeaways, as 
far as the Five Whys goes, it was a disaster. None of the people 
who were connected to and knew the most about the issues were 
at the meeting, and because this was the first time they were 
doing the Five Whys together, they didn’t stick to the format 
and went off on many tangents. It wasn’t a complete waste of 
time, but it didn’t have any of the benefits of the adaptive style 
of management discussed in this chapter. 


Don’t Send Your Baggage through the Five Whys Process 
IGN had the experience of trying to solve all of its “baggage” is- 


sues that had been causing wasted time for many years. Because 
this is an overwhelming set of problems, finding fixes quickly 
proves overwhelming. 

In their zeal to get started with the Five Whys, IGN ne- 
glected three important things: 


1. To introduce Five Whys to an organization, it is necessary 
to hold Five Whys sessions as new problems come up. 
Since baggage issues are endemic, they naturally come up 
as part of the Five Whys analysis and you can take that 
opportunity to fix them incrementally. If they don't come 
up organically, maybe they're not as big as they seem. 

2. Everyone who is connected to a problem needs to be at the 
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Five Whys session. Many organizations face the tempta- 
tion to save time by sparing busy people from the root 
cause analysis. This is a false economy, as IGN discovered 
the hard way. 

3. At the beginning of each Five Whys session, take a few 
minutes to explain what the process is for and how it 
works for the benefit of those who are new to it. If pos- 
sible, use an example of a successful Five Whys session 
from the past. If you're brand new, you can use my earlier 
example about the manager who doesn’t believe in train- 
ing. IGN learned that, whenever possible, it helps to use 
something that has personal meaning for the team. 


After our meeting, the IGN leadership decided to give Five 
Whys another try. Following the advice laid out in this chapter, 
they appointed a Five Whys master named Tony Ford, a direc- 
tor of engineering. Tony was an entrepreneur who had come to 
IGN through an acquisition. He got his start with Internet tech- 
nology, building websites about video games in the late 1990s. 
Eventually that led to an opportunity at a startup, TeamXbox, 
where he served as the lead software developer. TeamXbox was 
acquired by IGN Entertainment in 2003, and since that time 
Tony has been a technologist, leader of innovation, and propo- 
nent of agile and lean practices there. 

Unfortunately, Tony started without picking a narrow prob- 
lem area on which to focus. This led to early setbacks and frus- 
tration. Tony relates, “As, the new master I wasn’t very good at 
traversing through the Five Whys effectively, and the problems 
we were trying to solve were not great candidates in the first 
place. As you can imagine, these early sessions were awkward 
and in the end not very useful. I was getting quite discouraged 
and frustrated.” This is a common problem when one tries to 
tackle too much at once, but it is also a consequence of the fact 
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that these skills take time to master. Luckily, Tony persevered: 
“Having a Five Whys master is critical in my opinion. Five Whys 
is easy in theory but difficult in practice, so you need someone 
who knows it well to shape the sessions for those who don't.” 

The turnaround came when Tony led a Five Whys session 
involving a project that had been missing its deadlines. The ses- 
sion was fascinating and insightful and produced meaningful 
proportional investments. Tony explains: “The success had to 
do with a more experienced master and more experienced at- 
tendees. We all knew what the Five Whys was, and I did a really 
good job keeping us on track and away from tangents. This was 
a pivotal moment. Right then I knew the Five Whys was a new 
tool that was going to have a real impact on our overall success 
as a team and as a business.” 

On the surface, Five Whys seems to be about vochindcal prob- 
lems and preventing mistakes, but as teams drive out these su- 
perficial wastes, they develop a new understanding of how to 
work together. Tony put it this way: “I daresay that I discovered 
that the Five Whys transcends root cause analysis by revealing 
information that brings your team closer through a common 
understanding and perspective. A lot of times a problem can 
pull people apart; Five Whys does the opposite.” 

I asked Tony to provide an example of a recent successful Five 
Whys analysis from IGN. His account of it is listed in the sidebar. 


\e hy couldn't you add. or edit posts on the blogs? | | 
a Answer: } f A ny post request (write) to the article content tapi was S 
4 ‘returning a 500 error. 

~~ Proportional. investment: im —Wellw viork 6 on the API, but let? gs. 
make our CMS more forgiving for the user. Allow users toadd 
ae and edit drafts without ¢ errors for a better u user pesperience.. | 
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Why was the content API returning 500 errors? 

Answer: The bson_ext gem was incompatible with other gems 
it depends upon. 

Proportional investment: King—Remove the gem (already 
done to resolve the outage). 


Why was the gem incompatible? a 
Answer: We added a new version of the gem in addition to 
the existing version and the app started using it unexpectedly. 
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Proportional investment: Bennett—Convert our rails app to 


use bundler for gem.management. 


Why did we add a new version of a gem in production 
without testing? 

Answer: We didn’t think we needed a test in these cases. 
Proportional investment: Bennett and Jim—Write a unit or 
functional test in the API and CMS that will catch this in the 
future. 


Why do we add additional gems that we don’t intend to 
use right away? _ 
Answer: In preparation for a code push we wanted to get all 
“new gems ready in the production environment. Even though 
our code deployments are fully automated, gems are not. 
Proportional investment: Bennett—Automate gem manage- 
ment and installation into Continuous Integration and Con- 
tinuous Deployment process. 


Bonus—Why are we doing things in production on 
Friday nights? 


Answer: Because No one says we cant and it was a convenient 
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time for the developer to ‘prepare for a deployment wed be 
doing on Monday. 7 
Proportional investment: Tony—Make an announcement to — 
the team. There will be no production changes on Friday, — 
Saturday, or Sunday unless an exception has been made and 
approved by David (VP Engineering). We will reevaluate this 
policy when we have a fully automated continuous deploy- 
ment process in place. 


As a result of this Five Whys session and the proportional in- 
vestments we made, our deployments are easier, quicker, and 
never again will our process allow a developer to place gems 
into production systems with unintended consequences. In- 
deed, we have not had another issue like this. We strength- 
ened our “cluster immune system” as you would say. 
Without the Five Whys, we would have never discovered 
all of the information we did here. My guess is that we would 
have told that one developer to not do stupid things on Fri- 
day nights and moved on. This is what I emphasized earlier, 
where a-good Five Whys session has two outputs, learning and 
doing. The proportional investments that came out of this ses- 
sion are obviously valuable, but the learnings are much more 
subtle, but amazing for growing as developers and as a team. 


ADAPTING TO SMALLER BATCHES 


Before leaving the topic of building an adaptive organization, I 
want to introduce one more story. This one concerns a product 
that you've probably used if you've ever run your own business. 


It’s called QuickBooks, and it is one of Intuit’s flagship products. 
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QuickBooks has been the leading product in its category 
for many years. As a result, it has a large and dedicated cus- 
tomer base, and Intuit expects it to contribute significantly to 
its bottom line. Like most personal computer (PC) software 
of the last two decades, QuickBooks has been launched on 
an annual cycle, in one giant batch. This was how it worked 
three years ago, when Greg Wright, the director of product 
marketing for QuickBooks, joined the team. As you can imag- 
ine, there were lots of existing processes in ‘place to ensure a 
consistent product and an on-time release. The typical release 
approach was to spend significant up-front time to identify the 
customers’ need: 


Typically the first three to four months of each annual 
cycle was spent strategizing and planning, without build- 
ing new features. Once a plan and milestones were estab- 
lished, the team would spend the next six to nine months 
building. This would culminate in a big launch, and then 
the team would get its first feedback on whether it had 
successfully delivered on customers’ needs at the end of 
the process. 

So here was the time line: start process in September, 
first beta release is in June, second beta is in July. The beta 
is essentially testing to make sure it doesn’t crash people’s 
computers or cause them to lose their data—by that time 
in the process, only major bugs can be fixed. The design 
of the product itself is locked. 


This is the standard “waterfall” development methodology 
that product development teams have used for years. It is a lin- 
ear, large-batch system that relies for success on proper forecast- 
ing and planning. In other words, it is completely maladapted 
for today’s rapidly changing business environment. 
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Year One: Achieving Failure 


Greg witnessed this breakdown in 2009, his first year on the 
QuickBooks team. That year, the company shipped an entirely 
new system in QuickBooks for online banking, one of its most 
important features. The team went through rounds of usability 
testing using mock-ups and nonfunctional prototypes, followed 
by significant beta testing using sample customer data. At the 
moment of the launch, everything looked good. 

The first beta release was in June, and customer feedback 
started coming in negative. Although customers were complain- 
ing, there wasn't sufficient cause to stop the release because it 
was technically flawless—it didn’t crash computers. At that 
point, Greg was in a bind. He had no way of knowing how the 
feedback would translate to real customer behavior in the mar- 
ket. Were these just isolated complaints, or part of a widespread 
problem? He did know one thing for sure, though: that his team 
could not afford to miss the deadline. 

When the product finally shipped, the results were isiaiidie It 
took customers four to five times longer to reconcile their bank- 
ing transactions than it had with the older version. In the end, 
Greg’s team had failed to deliver on the customer need they were 
trying to address (despite building the product to specification), 
and because the next release had to go through the same water- 
fall process, it took the team nine months to fix. This is a classic 
case of “achieving failure’—-successfully executing a flawed plan. 

Intuit uses a tracking survey called the Net Promoter Score” 
to evaluate customer satisfaction with its many products. This is 
a great source of actionable metrics about what customers really 
think about a product. In fact, I used it at IMVU, too. One thing 
that is nice about NPS is that it is very stable over time. Since 
it is measuring core customer satisfaction, it is not subject to 
minor fluctuations; it registers only major changes in customer 
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sentiment. That year, the QuickBooks score dropped 20 points, 
the first time the company had ever moved the needle with the 
Net Promoter Score. That 20-point drop resulted in significant 
losses for Intuit and was embarrassing for the company—all be- 
cause customer feedback came too late in the process, allowing 
no time to iterate. 

Intuit’s senior management, including the general manager 
of the small business division and the head of small business 
accounting, recognized the need for change. To their credit, 
they tasked Greg with driving that change. His mission: to 
achieve startup speed for the development and deployment of 


QuickBooks. 


Year Two: Muscle Memory 


The next chapter of this story illustrates how hard it is to build 
an adaptive organization. Greg set out to change the Quick- 
Books development process by using four principles: 


1. Smaller teams. Shift from large teams with uniform func- 
tional roles to smaller, fully engaged teams whose mem- 
bers take on different roles. 

2. Achieve shorter cycle times. 

3. Faster customer feedback, testing both whether it crashes 
customers computers and the performance of new fea- 
tures/customer experience. 

4, Enable and empower teams to make fast and courageous 
decisions. 


On the surface, these goals seem to be aligned with the meth- 
ods and principles described in previous chapters, but Greg's 
second year with QuickBooks was not a marked success. For 
example, he decreed that the team would move to a midyear 
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release milestone, effectively cutting the cycle time and batch 
size in half. However, this was not successful. Through sheer de- 
termination, the team tried valiantly to get an alpha release out 
in January. However, the problems that afflict large-batch devel- 
opment were still present, and the team struggled to complete 
the alpha by April. That represented an improvement over the 
past system because issues could be brought to the surface two 
months earlier than under the old way, but it did not produce 
the dramatically better results Greg was looking for. 

In fact, over the course of the year, the team’s process kept 
looking more and more like it had in prior years. As Greg put it, 
“Organizations have muscle memory,” and it is hard for people 
to unlearn old habits. Greg was running up against a system, 
and making individual changes such as arbitrarily changing the 
release date were no match for it. 


Year Three: Explosion 


Frustrated by the limited progress in the previous year, Greg 
teamed up with the product development leader Himanshu 
Baxi. Together they tossed out all the old processes. They made 
a public declaration that their combined teams would be creat- 
ing new processes and that they were not going to go back to 
the old way. 

Instead of focusing on new deadlines, Greg and Himanshu 
invested in process, product, and technology changes that en- 
abled working in smaller batches. Those technical innovations 
helped them get the desktop product to customers faster for 
feedback. Instead of building a comprehensive road map at the 
beginning of the year, Greg kicked off the year with what they 
called idea/code/solution jams that brought engineers, product 
managers, and customers together to create a pipeline of ideas. 
It was scary for Greg as a product manager to start the year 
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without a defined list of what would be in the product release, 
but he had confidence in his team and the new process. 
There were three differences in year three: 


e Teams were involved in creating new technologies, pro- 
cesses, and systems. 

¢ Cross-functional teams were formed around new great 
ideas. 

¢ Customers were involved from the inception of each fea- 
ture concept. 


It’s important to understand that the old approach did not 
lack customer feedback or customer involvement in the plan- 
ning process. In the true spirit of genchi gembutsu, Intuit product 
managers (PMs) would do “follow-me-homes” with customers 
to identify problems to solve in the next release. However, the 
PMs were responsible for all the customer research. They would 
bring it back to the team and say, “This is the problem we want 
to solve, and here are ideas for how we could solve it.” - 

Changing to a cross-functional way of working was not 
smooth sailing. Some team members were skeptical. For ex- 
ample, some product managers felt that it was a waste of time 
for engineers to spend time in front of customers. The PMs 
thought that their job was to figure out the customer issue and 
define what needed to be built. Thus, the reaction of some PMs 
to the change was: “What's my job? What am I supposed to be 
doing?” Similarly, some on the engineering side just wanted to 
be told what to do; they didn’t want to talk to customers. As is 
typically the case in large-batch development, both groups had 
been willing to sacrifice the team’s ability to learn in order to 
work more “efficiently.” 

Communication was critical for this change process to suc- 
ceed. All the team leaders were open about the change they were 
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driving and why they were driving it. Much of the skepticism 
they faced was based on the fact that they did not have concrete 
examples of where this had worked in the past; it was an entirely 
new process for Intuit. They had to explain clearly why the old © 
process didn't work and why the annual release “train” was not 
setting them up for success. Throughout the change they com- 
municated the process outcomes they were shooting for: ear- 
lier customer feedback and a faster development cycle that was 
decoupled from the annual release time line. They repeatedly 
emphasized that the new approach was how startup competi- 
tors were working and iterating. They had to follow suit or risk 
becoming irrelevant. 
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Historically, QuickBooks had been built with large teams and 
long cycle times. For example, in earlier years the ill-fated online 
banking team had been composed of fifteen engineers, seven © 
quality assurance specialists, a product manager, and at times 
more than one designer. Now no team is bigger than five people. 
_ The focus of each team is iterating with customers as rapidly as 
possible, running experiments, and then using validated learn- 
ing to make real-time investment decisions about what to work 
on. As a result, whereas they used to have five major “branches” 
of QuickBooks that merged features at the time of the launch, 
now there are twenty to twenty-five branches. This allows for a 
much larger set of experiments. Each team works on a new fea- 
ture for approximately six weeks end to end, testing it with real 
customers throughout the process. 

Although the primary changes that are required in an adap- 
tive organization are in the mind-set of its employees, changing 
the culture is not sufficient. As we saw in Chapter 9, lean man- 
agement requires treating work as a system and then dealing 
with the batch size and cycle time of the whole process. Thus, 
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to achieve lasting change, the QuickBooks team had to invest 
in tools and platform changes that would enable the new, faster 
way of working. 

For example, one of the major stress points in the attempt to 
release an early alpha version the previous year was that Quick- 
Books is a mission-critical product. Many small businesses use it _ 
as their primary repository for critical financial data. The team 
was extremely wary of releasing a minimum viable product that 
had any risk of corrupting customer data. Therefore, even if they 
worked in smaller teams with a smaller scope, the burden of all 
that risk would have made it hard to work in smaller batches. 

To get the batch size down, the QuickBooks team had to 
invest in new technology. They built a virtualization system that 
allowed them to run multiple versions of QuickBooks on a cus- 
tomers computer. The second version could access all the cus- 
tomer's data but could not make permanent changes to it. Thus, 
there was no risk of the new version corrupting the customer's 
data by accident. This allowed them to isolate new releases to 
allow selected real customers to test them and provide feedback. 

The results in year three were promising. The version of 
QuickBooks that shipped that year had significantly higher cus- 
tomer satisfaction ratings and sold more units. If you're using 
QuickBooks right now, odds are you are using a version that 
was built in small batches. As Greg heads into his fourth year 
with the QuickBooks team, they are exploring even more ways 
to drive down batch size and cycle time. As usual, there are pos- 
sibilities that go beyond technical solutions. For example, the 
annual sales cycle of boxed desktop software is a significant bar- 
rier to truly rapid learning, and so the team has begun experi- 
menting with subscription-based products for the most active 
customers. With customers downloading updates online, Intuit 
can release software on a more frequent basis. Soon this program 
will see the QuickBooks team releasing to customers quarterly.° 
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As Lean Startups grow, they can use adaptive techniques to de- 
velop more complex processes without giving up their core ad- 
vantage: speed through the Build-Measure-Learn feedback loop. 
In fact, one of the primary benefits of using techniques that are 
derived from lean manufacturing is that Lean Startups, when 
they grow up, are well positioned to develop operational excel- 
lence based on lean principles. They already know how to op- 
erate with discipline, develop processes that are tailor-made to 
their situation, and use lean techniques such as the Five Whys 
and small batches. As a successful startup makes the transition 
to an established company, it will be well poised to develop the 
kind of culture of disciplined execution that characterizes the 
world’s best firms, such as Toyota. 

However, successfully growing into an established company 
is not the end of the story. A startup’s work is never done, be- 
cause as was discussed in Chapter 2, even established companies 
must struggle to find new sources of growth through disrup- 
tive innovation. This imperative is coming earlier in companies 
lives. No longer can a successful startup expect to have years 
after its initial public offering to bask in market-leading success. 
Today successful companies face immediate pressure from new 
competitors, fast followers, and scrappy startups. As a result, it 
no longer makes sense to think of startups as going through dis- 
crete phases like the proverbial metamorphosis of a caterpillar to 
a butterfly. Both successful startups and established companies _ 
alike must learn to juggle multiple kinds of work at the same 
time, pursuing operational excellence and disruptive innova- 
tion. This requires a new kind of portfolio thinking, which is 
the subject of Chapter 12. 7 


I? 
INNOVATE 


Sensei wisdom holds that when companies become larger, 
they inevitably lose the capacity for innovation, creativity, 
and growth. I believe this is wrong. As startups grow, entre- 
preneurs can build organizations that learn how to balance the 
needs of existing customers with the challenges of finding new 
customers to serve, managing existing lines of business, and ex- 
ploring new business models—all at the same time. And, if they 
are willing to change their management philosophy, I believe 
even large, established companies can make this shift to what I 


call portfolio thinking. 


HOW TO NURTURE DISRUPTIVE INNOVATION 


Successful innovation teams must be structured correctly in order 
to succeed. Venture-backed and bootstrapped startups naturally 
have some of these structural attributes as a consequence of being 
small, independent companies. Internal startup teams require 
support from senior management to create these structures. In- 
ternal or external, in my experience startup teams require three 
structural attributes: scarce but secure resources, independent 
authority to develop their business, and a personal stake in the 
outcome. Each of these requirements is different from those of 
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established company divisions. Keep in mind that structure is 
merely a prerequisite—it does not guarantee success. But getting 
the structure wrong can lead to almost certain failure. 


Scarce but Secure Resources 


Division leaders in large, established organizations are adept at 
using politics to enlarge their budgets but know that those bud- 
gets are somewhat loose. They often acquire as large a budget as 
possible and prepare to defend it against incursions from other 
departments. Politics means that they sometimes win and some- 
times lose: if a crisis emerges elsewhere in the organization, their 
budget might suddenly be reduced by 10 percent. This is not a 
catastrophe; teams will have to work harder and do more with 
less. Most likely, the budget has some padding in anticipation of 
this kind of eventuality. 

Startups are different: too much budget is as harmful as too 
littlke—as countless dot-com failures can attest—and startups 
are extremely sensitive to midcourse budgetary changes. It is ex- 
tremely rare for a stand-alone startup company to lose 10 per- 
cent of its cash on hand suddenly. In a large number of cases, 
this would be a fatal blow, as independent startups are run with 
little margin for error. Thus, startups are both easier and more 
demanding to run than traditional divisions: they require much 
less capital overall, but that capital must be absolutely secure 
from tampering. 


Independent Development Authority 


Startup teams need complete autonomy to develop and mar- 
ket new products within their limited mandate. They have to 
be able to conceive and execute experiments without having to 
gain an excessive number of approvals. 
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I strongly recommend that startup teams be completely 
cross-functional, that is, have full-time representation from 
every functional department in the company that will be in- 
volved in the creation or launch of their early products. They 
have to be able to build and ship actual functioning products 
and services, not just prototypes. Handoffs and approvals slow 
down the Build-Measure-Learn feedback loop and inhibit both 
learning and accountability. Startups require that they be kept 
to an absolute minimum. 

Of course, this level of development autonomy is liable to 
raise fears in a parent organization. Alleviating those fears is a 
major goal of the method recommended below. 


A Personal Stake in the Outcome 


Third, entrepreneurs need a personal stake in the outcome of their 
creations. In stand-alone new ventures, this usually is achieved 
through stock options or other forms of equity ownership. Where 
a bonus system must be used instead, the best incentives are tied 
to the long-term performance of the new innovation. 

However, I do not believe that a personal stake has to be 
financial. This is especially important in organizations, such as 
nonprofits and government, in which the innovation is not tied 
to financial objectives. In these cases, it is still possible for teams 
to have a personal stake. The parent organization has to make 
it clear who the innovator is and make sure the innovator re- 
ceives credit for having brought the new product to life—if it 
is successful. As one entrepreneur who ran her own division at 
a major media company told me, “Financial incentives aside, I 
always felt that because my name was on the door, I had more to 
lose and more to prove than someone else. That sense of owner- 
ship is not insignificant.” 

This formula is effective in for-profit companies as well. At 
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Toyota, the manager in charge of developing a new vehicle from 
start to finish is called the shusa, or chief engineer: 


Shusa are often called heavy-weight project managers in 
the U.S. literature, but this name understates their real 
roles as design leaders. Toyota employees translate the 
term as chief engineer, and they refer to the vehicle under 
development as the shusa’s car. They assured us that the 
shusa has final, absolute authority over every aspect of ve- 
hicle development.’ 


On the flip side, I know an extremely high-profile technol- 
ogy company that has a reputation for having an innovative 
culture, yet its track record of producing new products is disap- 
pointing. The company boasts an internal reward system that 
is based on large financial and status awards to teams that do 
something extraordinary, but those awards are handed out by 
senior management on the basis of—no one knows what. There 
are no objective criteria by which a team can gauge whether it 
will win this coveted lottery. Teams have little confidence that 
they will receive any long-term ownership of their innovations. 
Thus, teams rarely are motivated to take real risks, instead fo- 
cusing their energies on projects that are expected to win the 
approval of senior management. 


CREATING A PLATFORM FOR EXPERIMENTATION 


Next, it is important to focus on establishing the ground rules 
under which autonomous startup teams operate: how to protect 
the parent organization, how to hold entrepreneurial managers 
accountable, and how to reintegrate an innovation back into 
the parent organization if it is successful. Recall the “island of 
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freedom” that enabled the SnapTax team—in Chapter 2—to 
successfully create a startup within Intuit. That’s what a plat- 
form for experimentation can do. 


Protecting the Parent Organization 


Conventionally, advice about internal innovators focuses on 
protecting the startup from the parent organization. I believe it 
is necessary to turn this model on its head. , 

Let me begin by describing a fairly typical meeting from one 
of my consulting clients, a large company. Senior management 
had gathered to make decisions about what to include in the 
next version of its product. As part of the company’s commit- 
ment to being data-driven, it had tried to conduct an experi- 
ment on pricing. The first part of the meeting was taken up with 
interpreting the data from the experiment. 

One problem was that nobody could agree on what the data 
meant. Many custom reports had been created for the meeting; 
the data warehouse team was at the meeting too. The more they 
were asked to explain the details of each row on the spreadsheet, 
the more evident it became that nobody understood how those 
numbers had been derived. What we were left looking at was 
the number of gross sales of the product at a variety of different 
price points, broken down by quarter and by customer segment. 
It was a lot of data to try to comprehend. 

Worse, nobody was sure which customers had been exposed 
to the experiment. Different teams had been responsible for 
implementing it, and so different parts of the product had been 
updated at different times. The whole process had taken many 
months, and by this point, the people who had conceived the 
experiment had been moved to a division separate from that of 
the people who had executed it. 

You should be able to spot the many problems with this 
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situation: the use of vanity metrics instead of actionable metrics, 
an overly long cycle time, the use of large batch sizes, an unclear 
growth hypothesis, a weak experimental design, a lack of team 
ownership, and therefore very little learning. 

Listening in, I assumed this would be the end of the meeting. 
With no agreed-on facts to help make the decision, I thought 
nobody would have any basis for making the case for a particu- 
lar action. I was wrong. Each department simply took whatever 
interpretation of the data supported its position best and started 
advocating on its own behalf. Other departments would chime 
in with alternative interpretations that supported their positions, 
and so on. In the end, decisions were not made based on data. 
Instead, the executive running the meeting was forced to base 
decisions on the most plausible-sounding arguments. 

It seemed wasteful to me how much of the meeting had 
been spent debating the data because, in the end, the arguments 
that carried the day could have been made right at the start. 
It was as if each advocate sensed that he or she was about to 
be ambushed; if another team managed to bring clarity to the 
situation, it might undermine that person, and so the rational 
response was to obfuscate as much as possible. What a waste. 

Ironically, meetings like this had given data-driven decision 
making and experimentation a bad name inside the company, 
and for good reason. The data warehousing team was produc- 
ing reports that nobody read or understood. The project teams 
felt the experiments were a waste of time, since they involved 
building features halfway, which meant they were never any 
good. “Running an experiment” seemed to them to be code for 
postponing a hard decision. Worst of all, the executive team ex- 
perienced the meetings as chronic headaches. Their old prod- 
uct prioritization meetings might have been little more than a 
battle of opinions, but at least the executives understood what 
was going on. Now they had to go through a ritual that involved 
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complex math and reached no definite outcome, and then they 
ended up having a battle of opinions anyway. | | 


Rational Fears 


However, at the heart of this departmental feud was a very 
rational fear. This company served two customer segments: a 
business-to-business enterprise segment and a consumer seg- 
ment. In the B2B segment, the company employed sales staff 
to sell large volumes of the product to other companies, whereas 
the consumer segment was driven mostly by one-off purchases 
made by individuals. The bulk of the company’s current revenue 
came from B2B sales, but growth in that segment had been slow- 
ing. Everyone agreed there was tremendous potential for growth 
in the consumer segment, but so far little had materialized. 

Part of the cause of this lack of growth was the current pric- 
ing structure. Like many companies that sell to large enterprises, 
this one published a high list price and then provided heavy 
discounts to “favored” corporate clients who bought in bulk. 
Naturally, every salesperson was encouraged to make all of his or 
her clients feel favored. Unfortunately, the published list price 
was much too high for the consumer segment. 

The team in charge of growing the consumer segment wanted 
to run experiments with a lower price structure. The team in 
charge of the enterprise segment was nervous that this would 
cannibalize or otherwise diminish its existing relationships with 
its customers. What if those customers discovered that individu- 
als were getting a lower price than they were? 

Anyone who has been in a multisegment business will rec- 
ognize that there are many possible solutions to this problem, 
such as creating tiered feature sets so that different customers are 
able to purchase different “levels” of the product (as in airline 
seating) or even supporting different products under separate 
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brand names. Yet the company was struggling to implement any 
of those solutions. Why? Out of fear of endangering the current 
business, each proposed experiment would be delayed, sabo- 
taged, and obfuscated. 

It’s important to emphasize that this fear is well founded. 
Sabotage is a rational response from managers whose territory 
is threatened. This company is not a random, tiny startup with 
nothing to lose. An established company has a lot to lose. If the 
revenue from the core business goes down, heads will roll. This 
is not something to be taken lightly. 


The Dangers of Hiding Innovation inside the Black Box 


The imperative to innovate is unrelenting. Without the ability 
to experiment in a more agile manner, this company eventu- 
ally would suffer the fate described in The Innovators Dilemma: 
ever-higher profits and margins year after yeat until the business 
suddenly collapsed. 

We often frame internal innovation challenges by asking, 
How can we protect the internal startup from the parent orga- 
nization? I would like to reframe and reverse the question: 
How can we protect the parent organization from the startup? 
In my experience, people defend themselves when they feel 
threatened, and no innovation can flourish if defensiveness is 
given free rein. In fact, this is why the common suggestion to 
hide the innovation team is misguided. There are examples of 
one-time successes using a secret skunkworks or off-site inno- 
vation team, such as the building of the original IBM PC in 
Boca Raton, Florida, completely separate from mainline IBM. 
But these examples should serve mostly as cautionary tales, 
because they have rarely led to sustainable innovation.” Hid- 
ing from the parent organization can have long-term negative 
consequences. 
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Consider it from the point of view of the managers who have 
the innovation sprung on them. They are likely to feel betrayed 
and more than a little paranoid. After all, if something of this 
magnitude could be hidden, what else is waiting in the shadows? 
Over time, this leads to more politics as managers are incentiv- 
ized to ferret out threats to their power, influence, and careers. 
The fact that the innovation was a success is no justification for 
this dishonest behavior. From the point of view of established 
managers, the message is clear: if you are not on the inside, you 
are liable to be blindsided by this type of secret. 

It is unfair to criticize these managers for their response; the 
criticism should be aimed at senior executives who failed to 
design a supportive system in which to operate and innovate. 
I believe this is one reason why companies such as IBM lost 
their leadership position in the new markets that they devel- 
oped using a black box such as the PC business; they are unable 
to re-create and sustain the culture that led to the innovation in 


the first place. 


Creating an Innovation Sandbox 


The challenge here is to create a mechanism for empowering 
innovation teams out in the open. This is the path toward a 
sustainable culture of innovation over time as companies face 
repeated existential threats. My suggested solution is to create a 
sandbox for innovation that will contain the impact of the new 
innovation but not constrain the methods of the startup team. 
It works as follows: 


1. Any team can create a true split-test experiment that af- 
fects only the sandboxed parts of the product or service 
(for a multipart product) or only certain customer seg- 
ments or territories (for a new product). However: 
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2. One team must see the whole experiment through from 
end to end. 

3. No experiment can run longer than a specified amount of 
time (usually a few weeks for simple feature experiments, 
longer for more disruptive innovations). 

4. No experiment can affect more than a specified number 
of customers (usually expressed as a percentage of the 
company’s total mainstream customer base). 

5. Every experiment has to be evaluated on the basis of a 
single standard report of five to ten (no more) actionable 
metrics. 

6. Every team that works inside the sandbox and every 
product that is built must use the same metrics to evalu- 
ate success. 

7. Any team that creates an experiment must monitor the 
metrics and customer reactions (support calls, social 
media reaction, forum threads, etc.) while the experi- 
ment is in progress and abort it if something catastrophic 


happens. 


At the beginning, the sandbox has to be quite small. In the 
company above, the sandbox initially contained only the pricing 
page. Depending on the types of products the company makes, 
the size of the sandbox can be defined in different ways. For 
example, an online service might restrict it to certain pages or 
user flows. A retail operation might restrict it to certain stores 
or geographic areas. Companies trying to bring an entirely new 
product to market might build the restriction around customers 
in certain segments. 

Unlike in a concept test or market test, customers in the 
sandbox are considered real and the innovation team is allowed 
to attempt to establish a long-term relationship with them. After 
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all, they may be experimenting with those early adopters for a 
long time before their learning milestones are accomplished. 

Whenever possible, the innovation team should be cross- 
functional and have a clear team leader, like the Toyota shusa. 
It should be empowered to build, market, and deploy products 
or features in the sandbox without prior approval. It should be 
required to report on the success or failure of those efforts by 
using standard actionable metrics and innovation accounting. 

This approach can work even for teams that have never be- 
fore worked cross-functionally. The first few changes, such as a 
price change, may not require great engineering effort, but they 
require coordination across departments: engineering, market- 
ing, Customer service. Teams that work this way are more pro- 
ductive as long as productivity is measured by their ability to 
create customer value and not just stay busy. 

True experiments are easy to classify as successes or failures 
because top-level metrics either move or they don't. Either way, 
the team learns immediately whether its assumptions about how 
customers will behave are correct. By using the same metrics 
each time, the team builds literacy about those metrics across 
the company. Because the innovation team is reporting on its 
progress by using the system of innovation accounting described 
in Part Iwo, anyone who reads those reports is getting an im- 
plicit lesson in the power of actionable metrics. This effect is 
extremely powerful. Even if someone wants to sabotage the in- 
novation team, he or she will have to learn all about actionable 
metrics and learning milestones to do it. 

The sandbox also promotes rapid iteration. When people 
have a chance to see a project through from end to end and 
the work is done in small batches and delivers a clear verdict 
quickly, they benefit from the power of feedback. Each time 
they fail to move the numbers, they have a real opportunity 
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to act on their findings immediately. Thus, these teams tend 
to converge on optimal solutions rapidly even if they start out 
with really bad ideas. 

As we saw earlier, this is a manifestation of the principle of 
small batches. Functional specialists, especially those steeped in 
waterfall or stage-gate development, have been trained to work 
in extremely large batches. This causes even good ideas to get 
bogged down by waste. By making the batch size small, the 
sandbox method allows teams to make cheap mistakes quickly 
and start learning. As we'll see below, these small initial experi- 
ments can demonstrate that a team has a viable new business 
that can be integrated back into the parent company. 


Holding Internal Teams Accountable 


We already discussed learning milestones in detail in Chapter 7. 
With an internal startup team, the sequence of accountability is 
the same: build an ideal model of the desired disruption that is 
based on customer archetypes, launch a minimum viable prod- 
uct to establish a baseline, and then attempt to tune the engine 
to get it closer to the ideal. 

Operating in this framework, internal teams essentially act 
as startups. As they demonstrate success, they need to become 
integrated into the company’s overall portfolio of products and 
services. 


CULTIVATING THE MANAGEMENT PORTFOLIO 


There are four major kinds of work that companies must man- 
age.” As an internal startup grows, the entrepreneurs who cre- 
ated the original concept must tackle the challenge of scale. As 
new mainstream customers are acquired and new markets are 
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conquered, the product becomes part of the public face of the 
company, with important implications for PR, marketing, sales, 
and business development. In most cases, the product will at- 
tract competitors: copycats, fast followers, and imitators of all 
stripes. 

Once the market for the new product is well established, 
procedures become more routine. To combat the inevitable 
commoditization of the product in its market, line extensions, 
incremental upgrades, and new forms of marketing are essential. 
In this phase, operational excellence takes on a greater role, as an 
important way to increase margins is to lower costs. This may 
require a different type of manager: one who excels in optimiza- 
tion, delegation, control, and execution. Company stock prices 
depend on this kind of predictable growth. 

There is a fourth phase as well, one dominated by operating 
costs and legacy products. This is the domain of outsourcing, 
automation, and cost reduction. Nonetheless, infrastructure 
is still mission-critical. Failure of facilities or important infra- 
structure or the abandonment of loyal customers could derail 
the whole company. However, unlike the growth and optimiza- 
tion phase, investments in this area will not help the company 
achieve top-line growth. Managers of this kind of organization 
suffer the fate of baseball umpires: criticized when something 
goes wrong, unappreciated when things are going well. 

We tend to speak of these four phases of businesses from the 
perspective of large companies, in which they may represent en- 
tire divisions and hundreds or even thousands of people. That’s 
logical, as the evolution of the business in these kinds of extreme 
cases is the easiest to observe. However, all companies engage in 
all four phases of work all the time. As soon as a product hits 
the marketplace, teams of people work hard to advance it to 
the next phase. Every successful product or feature began life in 
research and development (R&D), eventually became a part of 
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the company’s strategy, was subject to optimization, and in time 
became old news. 

The problem for startups and large companies alike is that em- 
ployees often follow the products they develop as they move from 
phase to phase. A common practice is for the inventor of a new 
product or feature to manage the subsequent resources, team, or 
division that ultimately commercializes it. As a result, strong cre- 
ative managers wind up getting stuck working on the growth and 
optimization of products rather than creating new ones. 

This tendency is one of the reasons established compa- 
nies struggle to find creative managers to foster innovation in 
the first place. Every new innovation competes for resources 
with established projects, and one of the scarcest resources is 
talent. 


Entrepreneur Is a Job Title 


The way out of this dilemma is to manage the four kinds of work 
differently, allowing strong cross-functional teams to develop 
around each area. When products move from phase to phase, 
they are handed off between teams. Employees can choose to 
move with the product as part of the handoff or stay behind 
and begin work on something new. Neither choice is necessarily 
right or wrong; it depends on the temperament and skills of the 
person in question. 

Some people are natural inventors who prefer to work with- 
out the pressure and expectations of the later business phases. 
Others are ambitious and see innovation as a path toward se- 
nior management. Still others are particularly skilled at the 
management of running an established business, outsourcing, 
and bolstering efficiencies and wringing out cost reductions. 
People should be allowed to find the kinds of jobs that suit 
them best. 
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In fact, entrepreneurship should be considered a viable ca- 
reer path for innovators inside large organizations. Managers 
who can lead teams by using the Lean Startup methodology 
should not have to leave the company to reap the rewards of 
their skills or have to pretend to fit into the rigid hierarchies of 
established functional departments. Instead, they should have a 
business card that says simply “Entrepreneur” under the name. 
They should be held accountable via the system of innovation 
accounting and promoted and rewarded accordingly. 

After an entrepreneur has incubated a product in the inno- 
vation sandbox, it has to be reintegrated into the parent orga- 
nization. A larger team eventually will be needed to grow it, 
commercialize it, and scale it. At first, this team will require the 
continued leadership of the innovators who worked in the sand- 
box. In fact, this is a positive part of the process in that it gives 
the innovators a chance to train new team members in the new 
style of working that they mastered in the original sandbox. 

Ideally, the sandbox will grow over time; that is, rather than 
move the team out of the sandbox and into the company’s stan- 
dard routines, there may be opportunities to enlarge the scope of 
the sandbox. For example, if only certain aspects of the product 
were subject to experimentation in the sandbox, new features 
can be added. In the online service described earlier, this could 
be accomplished by starting with a sandbox that encompassed 
the product pricing page. When those experiments succeeded, 
the company could add the website’s home page to the sandbox. 
It subsequently might add the search functionality or the over- 
all web design. If only certain customers or certain numbers of 
customers were targeted initially, the product’s reach could be 
increased. When such changes are contemplated, it’s important 
that senior management consider whether the teams working 
in the sandbox can fend for themselves politically in the parent 
organization. The sandbox was designed to protect them and 
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the parent organization, and any expansion needs to take this 
into account. | : 

Working in the innovation sandbox is like developing startup 
muscles. At first, the team will be able to take on only modest 
experiments. The earliest experiments may fail to produce much 
learning and may not lead to scalable success. Over time, those 
teams are almost guaranteed to improve as long as they get the 
constant feedback of small-batch development and actionable 
metrics and are held accountable to learning milestones. 

Of course, any innovation system eventually will become the 
victim of its own success. As the sandbox expands and the com- 
pany’s revenue grows as a result of the sandbox’s innovations, the 
cycle will have to begin again. The former innovators will be- 
come guardians of the status quo. When the product makes up 
the whole sandbox, it inevitably will become encumbered with 
the additional rules and controls needed for mission-critical op- 
eration. New innovation teams will need a new sandbox within 


which to play. 


Becoming the Status Quo 


This last transition is especially hard for innovators to accept: 
their transformation from radical outsiders to the embodiment 
of the status quo. I have found it disturbing in my career. As 
you can guess from the techniques I advocate as part of the 
Lean Startup, I have always been a bit of a troublemaker at the 
companies at which I have worked, pushing for rapid iteration, 
data-driven decision making, and early customer involvement. 
When these ideas were not part of the dominant culture, it was 
simple (if frustrating) to be an advocate. All I had to do was 
push as hard as humanly possible for my ideas. Since the domi- 
nant culture found them heretical, they would compromise 
with me a “reasonable” amount. Thanks to the psychological 
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phenomenon of anchoring, this led to a perverse incentive: the 
more radical my suggestion was, the more likely it was that the 
reasonable compromise would be closer to my true goal. 

Fast-forward several years to when I was running product de- 
velopment. When we'd hire new people, they had to be indoc- 
trinated into the Lean Startup culture. Split testing, continuous 
deployment, and customer testing were all standard practice. J 
needed to continue to be a strong advocate for my ideas, making 
sure each new employee was ready to give them a try. But for 
the people who had been working there awhile, those ideas had 
become part of the status quo. 

Like many entrepreneurs, I was caught between constant 
-evangelizing for my ideas and constantly entertaining sugges- 
tions for ways they could be improved. My employees faced the 
same incentive I had exploited years before: the more radical 
the suggestion is, the more likely it is that the compromise will 
move in the direction they desire. I heard it all: suggestions that 
we go back to waterfall development, use more quality assur- 
ance (QA), use less QA, have more or less customer involve- 
ment, use more vision and less data, or interpret data in a more 
statistically rigorous way. 

It took a constant effort to consider these suggestions seri- 
ously. However, responding dogmatically is unhelpful. Com- 
promising by automatically splitting the difference doesn’t work 
either. | 

I've found that every suggestion should be subjected to the 
same rigorous scientific inquiry that led to the creation of the 
Lean Startup in the first place. Can we use the theory to predict 
the results of the proposed change? Can we incubate the change 
in a small team and see what happens? Can we measure its im- 
pact? Whenever they could be implemented, these approaches 
have allowed me to increase my own learning and, more im- 
portant, the productivity of the companies I have worked 
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with. Many of the Lean Startup techniques that we pioneered 
at IMVU are not my original contributions. Rather, they were 
conceived, incubated, and executed by employees who brought 
their own creativity and talent to the task. 

Above all, I faced this common question: How do we know 
that “your way” of building a company will work? What other 
companies are using it? Who has become rich and famous as a 
result? These questions are sensible. The titans of our industry 
are all working in a slower, more linear way. Why are we doing 
something different? 

It is these questions that require the use of theory to answer. 
Those who look to adopt the Lean Startup as a defined set of 
steps or tactics will not succeed. I had to learn this the hard way. 
In a startup situation, things constantly go wrong. When that 
happens, we face the age-old dilemma summarized by Deming: 
How do we know that the problem is due to a special cause 
versus a systemic cause? If we're in the middle of adopting a 
new way of working, the temptation will always be to blame 
the new system for the problems that arise. Sometimes that ten- 
_ dency is correct, sometimes not. Learning to tell the difference 
requires theory. You have to be able to predict the outcome of 
the changes you make to tell if the problems that result are really 
problems. 7 

For example, changing the definition of productivity for a 
team from functional excellence—excellence in marketing, 
sales, or product development—to validated learning will cause 
problems. As was indicated earlier, functional specialists are ac- 
customed to measuring their efficiency by looking at the pro- 
portion of time they are busy doing their work. A programmer 
expects to be coding all day long, for example. That is why many 
traditional work environments frustrate these experts: the con- 
stant interruption of meetings, cross-functional handoffs, and 
explanations for endless numbers of bosses all act as a drag on 
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efficiency. However, the individual efficiency of these specialists 
is not the goal in a Lean Startup. Instead, we want to force teams 
to work cross-functionally to achieve validated learning. Many 
of the techniques for doing this—actionable metrics, continu- 
ous deployment, and the overall Build-Measure-Learn feedback 
loop—necessarily cause teams to suboptimize for their individ- 
ual functions. It does not matter how fast we can build. It does 
not matter how fast we can measure. What matters is how fast 
we can get through the entire loop. 

In my years teaching this system, I have noticed this pattern 
every time: switching to validated learning feels worse before it 
feels better. That’s the case because the problems caused by the 
old system tend to be intangible, whereas the problems of the 
new system are all too tangible. Having the benefit of theory 
is the antidote to these challenges. If it is known that this loss 
of productivity is an inevitable part of the transition, it can be 
managed actively. Expectations can be set up front. In my con- 
sulting practice, for example, I have learned to raise these is- 
sues from day one; otherwise, they are liable to derail the whole 
effort once it is under way. As the change progresses, we can 
use the root cause analysis and fast response techniques to fig- 
ure out which problems need prevention. Ultimately, the Lean 
Startup is a framework, not a blueprint of steps to follow. It is 
designed to be adapted to the conditions of each specific com- 
pany. Rather than copy what others have done, techniques such 
as the Five Whys allow you to build something that is perfectly 
suited to your company. 

The best way to achieve mastery of and explore these ideas is 
to embed oneself in a community of practice. There is a thriving 
community of Lean Startup meetups around the world as well 
as online, and suggestions for how you can take advantage of 
these resources listed in the last chapter of this book, “Join the 
Movement.” 


13 
EPILOGUE: WASTE NOT 


his year marks the one hundredth anniversary of Frederick Wins- 

low Taylors The Principles of Scientific Management, first 
published in 1911. The movement for scientific management 
changed the course of the twentieth century by making possible 
the tremendous prosperity that we take for granted today. Taylor 
effectively invented what we now consider simply management: 
improving the efficiency of individual workers, management 
by exception (focusing only on unexpectedly good or bad re- 
sults), standardizing work into tasks, the task-plus-bonus system 
of compensation, and—above all—the idea that work can be 
studied and improved through conscious effort. Taylor invented 
modern white-collar work that sees companies as systems that 
must be managed at more than the level of the individual. There 
is a reason all past management revolutions have been led by 
engineers: management is human systems engineering. 

In 1911 Taylor wrote: “In the past, the man has been first; 
in the future, the system must be first.” Taylor’s prediction has 
come to pass. We are living in the world he imagined. And yet, 
the revolution that he unleashed has been—in many ways— 
too successful. Whereas Taylor preached science as a way of 
thinking, many people confused his message with the rigid 
techniques he advocated: time and motion studies, the differ- 
ential piece-rate system, and—most galling of all—the idea that 
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workers should be treated as little more than automatons. Many 
of these ideas proved extremely harmful and required the efforts 
of later theorists and managers to undo. Critically, lean manu- 
facturing rediscovered the wisdom and initiative hidden in every 
factory worker and redirected Taylor’s notion of efficiency away 
from the individual task and toward the corporate organism as 
a whole. But each of these subsequent revolutions has embraced 
‘Taylor's core idea that work can be studied scientifically and can 
be improved through a rigorous experimental approach. 

In the twenty-first century, we face a new set of problems 
that Taylor could not have imagined. Our productive capacity 
greatly exceeds our ability to know what to build. Although there 
was a tremendous amount of invention and innovation in the 
early twentieth century, most of it was devoted to increasing the 
productivity of workers and machines in order to feed, clothe, 
and house the world’s population. Although that project is still 
incomplete, as the millions who live in poverty can attest, the 
solution to that problem is now strictly a political one. We have 
the capacity to build almost anything we can imagine. The big 
question of our time is not Can it be built? but Should it be 
built? This places us in an unusual historical moment: our future 
prosperity depends on the quality of our collective imaginations. 

In 1911, Taylor wrote: 


We can see our forests vanishing, our water-powers going 
to waste, our soil being carried by floods into the sea; and 
the end of our coal and our iron is in sight. But our larger 
wastes of human effort, which go on every day through 
such of our acts as are blundering, ill-directed, or ineffi- 
cient . . . are less visible, less tangible, and are but vaguely 
appreciated. 

We can see and feel the waste of material things. 
Awkward, inefficient, or ill-directed movements of men, 
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however, leave nothing visible or tangible behind them. 
Their appreciation calls for an act of memory, an effort 
of the imagination. And for this reason, even though our 
daily loss from this source is greater than from our waste 
of material things, the one has stirred us deeply, while the 
other has moved us but little.’ 


A century on, what can we say about those words? On the 
one hand, they feel archaic. We of the twenty-first century are 
hyperaware of the importance of efficiency and the economic 
value of productivity gains. Our workplaces are—at least when 
it comes to the building of material objects—incredibly well or- 
ganized compared with those of Taylor's day. 

On the other hand, Taylor’s words strike me as completely 
contemporary. For all of our vaunted efficiency in the making 
of things, our economy is still incredibly wasteful. ‘This waste 
comes not from the inefficient organization of work but rather 
from working on the wrong things—and on an industrial 
scale. As Peter Drucker said, “There is surely nothing quite so 
useless as doing with great efficiency what should not be done 
at all.”? 

And yet we are doing the wrong things efficiently all the 
time. It is hard to come by a solid estimate of just how wasteful 
modern work is, but there is no shortage of anecdotes. In my 
consulting and travels talking about the Lean Startup, I hear 
the same message consistently from employees of companies 
big and small. In every industry we see endless stories of failed 
launches, ill-conceived projects, and large-batch death spirals. 
I consider this misuse of people's time a criminally negligent 
waste of human creativity and potential. 

What percentage of all this waste is preventable? I think a 
much larger proportion than we currently realize. Most people I 
meet believe that in their industry at least, projects fail for good 
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reasons: projects are inherently risky, market conditions are un- 
predictable, “big company people” are intrinsically uncreative. 
Some believe that if we just slowed everything down and used a 
more careful process, we could reduce the failure rate by doing 
fewer projects of higher quality. Others believe that certain peo- 
ple have an innate gift of knowing the right thing to build. If we 
can find enough of these visionaries and virtuosos, our problems 
will be solved. These “solutions” were once considered state of 
the art in the nineteenth century, too, before people knew about 
modern management. 

The requirements of an ever-faster world make these antique 
approaches unworkable, and so the blame for failed projects 
and businesses often is heaped on senior management, which is 
asked to do the impossible. Alternatively, the finger of blame is 
pointed at financial investors or the public markets for overem- 
phasizing quick fixes and short-term results. We have plenty of 
blame to go around, but far too little theory to guide the actions 
of leaders and investors alike. 

The Lean Startup movement stands in contrast to this 
hand-wringing. We believe that most forms of waste in innova- 
tion are preventable once their causes are understood. All that 
is required is that we change our collective mind-set concerning 
how this work is to be done. 

It is insufficient to exhort workers to try harder. Our current 
problems are caused by trying too hard—at the wrong things. 
By focusing on functional efficiency, we lose sight of the real 
goal of innovation: to learn that which is currently unknown. 
As Deming taught, what matters is not setting quantitative goals 
but fixing the method by which those goals are attained. The 
Lean Startup movement stands for the principle that the scien- 
tific method can be brought to bear to answer the most pressing 
innovation question: How can we build a sustainable organiza- 
tion around a new set of products or services? 
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ORGANIZATIONAL SUPERPOWERS 


A participant at one of my workshops came up to me a few 
months afterward to relate the following story, which I am para- 
phrasing: “Knowing Lean Startup principles makes me feel like 
I have superpowers. Even though I’m just a junior employee, 
when I meet with corporate VPs and GMs in my large com- 
pany, I ask them simple questions and very quickly help them 
see how their projects are based on fundamental hypotheses that 
are testable. In minutes, I can lay out a plan they could fol- 
low to scientifically validate their plans before it’s too late. They 
consistently respond with “Wow, you are brilliant. We've never 
thought to apply that level of rigor to our thinking about new 
products before.” 

As a result of these interactions, he has developed a reputa- 
tion within his large company as a brilliant employee. This has 
been good for his career but very frustrating for him personally. 
Why? Because although he is quite brilliant, his insights into 
flawed product plans are due not to his special intelligence but to 
having a theory that allows him to predict what will happen and 
propose alternatives. He is frustrated because the managers he is 
pitching his ideas to do not see the system. They wrongly con- 
clude that the key to success is finding brilliant people like him 
to put on their teams. They are failing to see the opportunity he 
is really presenting them: to achieve better results systematically 
by changing their beliefs about how innovation happens. 


Putting the System First: Some Dangers 


Like Taylor before us, our challenge is to persuade the manag- 
ers of modern corporations to put the system first. However, 
Taylorism should act as a cautionary tale, and it is important to 
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learn the lessons of history as we bring these new ideas to a more 
mainstream audience. 

Taylor is remembered for his focus on systematic practice 
rather than individual brilliance. Here is the full quote from The 
Principles of Scientific Management that includes the famous line 
about putting the system first: 


In the future it will be appreciated that our leaders must be 
trained right as well as born right, and that no great man 
can (with the old system of personal management) hope 
to compete with a number of ordinary men who have 
been properly organized so as efficiently to cooperate. 

In the past the man has been first; in the future the 
system must be first. This in no sense, however, implies 
that great men are not needed. On the contrary, the first 
object of any good system must be that of developing 
first-class men; and under systematic management the 
best man rises to the top more certainly and more rapidly 
than ever before.° 


Unfortunately, Taylor's insistence that scientific management 
does not stand in opposition to finding and promoting the best 
individuals was quickly forgotten. In fact, the productivity gains 
to be had through the early scientific management tactics, such 
as time and motion study, task-plus-bonus, and especially func- 
tional foremanship (the forerunner of today’s functional de- 
partments), were so significant that subsequent generations of 
managers lost sight of the importance of the people who were 
implementing them. 

This has led to two problems: (1) business systems became 
overly rigid and thereby failed to take advantage of the adapt- 
ability, creativity, and wisdom of individual workers, and (2) 
there has been an overemphasis on planning, prevention, and 


278 = THE LEAN STARTUP 


procedure, which enable organizations to achieve consistent 
results in a mostly static world. On the factory floor, these 
problems have been tackled head on by the lean manufactur- 
ing movement, and those lessons have spread throughout many 
modern corporations. And yet in new product development, 
entrepreneurship, and innovation work in general we are still 
using an outdated framework. 

My hope is that the Lean Startup movement will not fall into 
the same reductionist trap. We are just beginning to uncover the 
rules that govern entrepreneurship, a method that can improve 
the odds of startup success, and a systematic approach to build- 
ing new and innovative products. This in no way diminishes 
the traditional entrepreneurial virtues: the primacy of vision, the 
willingness to take bold risks, and the courage required in the 
face of overwhelming odds. Our society needs the creativity and 
vision of entrepreneurs more than ever. In fact, it is precisely 
because these are such precious resources that we cannot afford 
to waste them. 


Product Development Pseudoscience 


I believe that if Taylor were alive today, he would chuckle at 
what constitutes the management of entrepreneurs and innova- 
tors. Although we harness the labor of scientists and engineers 
who would have dazzled any early-twentieth-century person 
with their feats of technical wizardry, the management practices 
we use to organize them are generally devoid of scientific rigor. 
In fact, I would go so far as to call them pseudoscience. 

We routinely green-light new projects more on the basis of 
intuition than facts. As we've seen throughout this book, that is 
not the root cause of the problem. All innovation begins with 
vision. It’s what happens next that is critical. As we've seen, too 
many innovation teams engage in success theater, selectively 
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finding data that support their vision rather than exposing the 
elements of the vision to true experiments, or, even worse, stay- 
ing in stealth mode to create a data-free zone for unlimited “ex- 
perimentation” that is devoid of customer feedback or external 
accountability of any kind. Anytime a team attempts to demon- 
strate cause and effect by placing highlights on a graph of gross 
metrics, it is engaging in pseudoscience. How do we know that 
the proposed cause and effect is true? Anytime a team attempts 
to justify its failures by resorting to learning as an excuse, it is 
engaged in pseudoscience as well. 

If learning has taken place in one iteration cycle, let us dem- 
onstrate it by turning it into validated learning in the next cycle. 
Only by building a model of customer behavior and then show- 
ing our ability to use our product or service to change it over 
time can we establish real facts about the validity of our vision. 

Throughout our celebration of the success of the Lean Startup 
movement, a note of caution is essential. We cannot afford to 
have our success breed a new pseudoscience around pivots, 
MVPs, and the like. This was the fate of scientific management, 
and in the end, I believe, that set back its cause by decades. Sci- 
ence came to stand for the victory of routine work over creative 
work, mechanization over humanity, and plans over agility. Later 
movements had to be spawned to correct those deficiencies. 

Taylor believed in many things that he dubbed scientific but 
that our modern eyes perceive as mere prejudice. He believed 
in the inherent superiority in both intelligence and character 
of aristocratic men over the working classes and the superiority 
of men over women; he also thought that lower-status people 
should be supervised strictly by their betters. These beliefs are 
part and parcel of Taylor's time, and it is tempting to forgive 
him for having been blind to them. 

Yet when our time is viewed through the lens of future prac- 
tice, what prejudices will be revealed? In what forces do we place 
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undue faith? What might we risk losing sight of with this initial 
success of our movement? 

It is with these questions that I wish to close. As gratifying as it 
is for me to see the Lean Startup movement gain fame and recog- 
nition, it is far more important that we be right in our prescrip- 
tions. What is known so far is just the tip of the iceberg. What 
is needed is a massive project to discover how to unlock the vast 
stores of potential that are hidden in plain sight in our modern 
workforce. If we stopped wasting people’s time, what would they 
do with it? We have no real concept of what is possible. 

Starting in the late 1880s, Taylor began a program of experi- 
mentation to discover the optimal way to cut steel. In the course 
of that research, which lasted more than twenty-five years, he 
and his colleagues performed more than twenty thousand in- - 
dividual experiments. What is remarkable about this project is 
that it had no academic backing, no government R&D bud- 
get. Its entire cost was paid by industry out of the immediate 
profits generated from the higher productivity the experiments 
enabled. This was only one experimental program to uncover 
the hidden productivity in just one kind of work. Other scien- 
tific management disciples spent years investigating bricklaying, 
farming, and even shoveling. They were obsessed with learning 
the truth and were not satisfied with the folk wisdom of crafts- 
persons or the parables of experts. 

Can any of us imagine a modern knowledge-work manager 
with the same level of interest in the methods his or her employ- 
ees use? How much of our current innovation work is guided by 
catchphrases that lack a scientific foundation? 


A New Research Program 


What comparable research programs could we be engaged in to 
discover how to work more effectively? 
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For one thing, we have very little understanding of what 
stimulates productivity under conditions of extreme uncertainty. 
Luckily, with cycle times falling everywhere, we have many op- 
portunities to test new approaches. Thus, I propose that we cre- 
ate startup testing labs that could put all manner of product 
development methodologies to the test. 

How might those tests be conducted? We could bring in 
small cross-functional teams, perhaps beginning with product 
and engineering, and have them work to solve problems by 
using different development methodologies. We could begin 
with problems with clear right answers, perhaps drawn from the 
many international programming competitions that have devel- 
oped databases of well-defined problems with clear solutions. 
These competitions also provide a clear baseline of how long it 
should take for various problems to be solved so that we could 
establish clearly the individual problem-solving prowess of the 
experimental subjects. 

Using this kind of setup for calibration, we could begin to 
vary the conditions of the experiments. The challenge will be 
to increase the level of uncertainty about what the right answer 
is while still being able to measure the quality of the outcome 
objectively. Perhaps we could use real-world customer problems 
and then have real consumers test the output of the teams’ work. 
Or perhaps we could go so far as to build minimum viable prod- 
ucts for solving the same set of problems over and over again to 
quantify which produces the best customer conversion rates. 

We also could vary the all-important cycle time by choosing 
more or less complex development platforms and distribution 
channels to test the impact of those factors on the true produc- 
tivity of the teams. 

Most of all, we need to develop clear methods for holding 
teams accountable for validated learning. I have proposed one 
method in this book: innovation accounting using a well-defined 
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financial model and engine of growth. However, it is naive to as- 
sume that this is the best possible method. As it is adopted in 
more and more companies, undoubtedly new techniques will be 
suggested, and we need to be able to evaluate the new ideas as 
rigorously as possible. 

All these questions raise the possibilities of public-private 
partnerships between research universities and the entrepreneur- 
ial communities they seek to foster. It also suggests that uni- 
versities may be able to add value in more ways than by being 
simply financial investors or creators of startup incubators, as is 
the current trend. My prediction is that wherever this research 
is conducted will become an epicenter of new entrepreneurial 
practice, and universities conducting this research therefore may 
be able to achieve a much higher level of commercialization of 
their basic research activities.‘ 


THE LONG-TERM STOCK EXCHANGE 


Beyond simple research, I believe our goal should be to change 
the entire ecosystem of entrepreneurship. Too much of our 
startup industry has devolved into a feeder system for giant 
media companies and investment banks. Part of the reason es- 
tablished companies struggle to invest consistently in innova- 
tion is intense pressure from public markets to hit short-term 
profitability and growth targets. Mostly, this is a consequence 
of the accounting methods we have developed for evaluating 
managers, which focus on the kinds of gross “vanity” metrics 
discussed in Chapter 7. What is needed is a new kind of stock 
exchange, designed to trade in the stocks of companies that are 
organized to sustain long-term thinking. I propose that we cre- 
ate a Long-Term Stock Exchange (LTSE). 


In addition to quarterly reports on profits and margins, 
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companies on the LISE would report using innovation account- 
ing on their internal entrepreneurship efforts. Like Intuit, they 
would report on the revenue they were generating from products 
that did not exist a few years earlier. Executive compensation 
in LTSE companies would be tied to the company’s long-term 
performance. Trading on the LTSE would have much higher 
transaction costs and fees to minimize day trading and massive 
price swings. In exchange, LTSE companies would be allowed to 
structure their corporate governance to facilitate greater freedom 
for management to pursue long-term investments. In addition 
to support for long-term thinking, the transparency of the LTSE 
will provide valuable data about how to nurture innovation in 
the real world. Something like the LTSE would accelerate the 
creation of the next generation of great companies, built from 
the ground up for continuous innovation. 


IN CONCLUSION 


As a movement, the Lean Startup must avoid doctrines and 
rigid ideology. We must avoid the caricature that science means 
formula or a lack of humanity in work. In fact, science is one 
of humanity's most creative pursuits. I believe that applying 
it to entrepreneurship will unlock a vast storehouse of human 
potential. 

What would an organization look like if all of its employees 
were armed with Lean Startup organizational superpowers? 

For one thing, everyone would insist that assumptions be 
stated explicitly and tested rigorously not as a stalling tactic or a 
form of make-work but out of a genuine desire to discover the 
truth that underlies every project’s vision. 

We would not waste time on endless arguments between 
the defenders of quality and the cowboys of reckless advance; 
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instead, we would recognize that speed and quality are allies 
in the pursuit of the customer’s long-term benefit. We would 
race to test our vision but not to abandon it. We would look to 
eliminate waste not to build quality castles in the sky but in the 
service of agility and breakthrough business results. 

We would respond to failures and setbacks with honesty 
and learning, not with recriminations and blame. More than 
that, we would shun the impulse to slow down, increase batch 
size, and indulge in the curse of prevention. Instead, we would 
achieve speed by bypassing the excess work that does not lead 
_to learning. We would dedicate ourselves to the creation of new 
institutions with a long-term mission to build sustainable value 
and change the world for the better. 

Most of all, we would stop wasting people’s time. 


|4 
JOIN THE MOVEMENT 


nthe past few years, the Lean Startup movement has gone global. 

The number of resources available for aspiring entrepreneurs 
is incredible. Here, I'll do my best to list just a few of the best 
events, books, and blogs for further reading and further prac- 
tice. The rest is up to you. Reading is good, action is better. 

The most important resources are local. Gone are the days 
where you had to be in Silicon Valley to find other entrepreneurs 
to share ideas and struggles with. However, being embedded in 
a startup ecosystem is still an important part of entrepreneur- 
ship. What's changed is that these ecosystems are springing up 
in more and more startup hubs around the world. 

I maintain an official website for The Lean Startup at http:// 
theleanstartup.com, where you can find additional resources, in- 
cluding case studies and links to further reading. You will also 
find links there to my blog, Startup Lessons Learned, as well as 
videos, slides, and audio from my past presentations. 


Lean Startup Meetups 


Chances are there is a Lean Startup meetup group near you. 
As of this writing, there are over a hundred, with the largest in 


286 The Lean Startup 


San Francisco, Boston, New York, Chicago, and Los Angeles. 
You can find a real-time map of groups here: http://lean-startup 
-meetup.com/. You can also find a list of cities where people 
are interested in starting a new group, and tools to set one up 
yourself, 


The Lean Startup Wiki 


Not every Lean Startup group uses Meetup.com to organize, and a 
comprehensive list of events and other resources is maintained by 
volunteers on the Lean Startup Wiki: hetp://leanstartup.pbworks 
.com/ 


The Lean Startup Circle 


The largest community of practice around the Lean Startup is 
happening online, right now, on the Lean Startup Circle mailing 
list. Founded by, Rich Collins, the list has thousands of entrepre- 
neurs sharing tips, resources, and stories every day. If you have a 
question about how Lean Startup might apply to your business 
or industry, it's a great place to start: http://leanstartupcircle 
.com/ 


The Startup Lessons Learned Conference 


For the past two years, I have run a conference called Startup Les- 
sons Learned. More details are available here: http://sllconf.com 


REQUIRED READING 


Steve Blank’s book The Four Steps to the Epiphany is the origi- 


nal book about customer development. When I was building 
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IMVU, a dog-eared copy of this book followed me everywhere. 
It is an indispensable guide. You can get a copy here: http://ericri 
.es/FourSteps or read my review of it here: http://www.startup 
lessonslearned.com/2008/1 1/what-is-customer-development 
-html. Steve also maintains an active and excellent blog at http:// 
steveblank.com/ | 

Brant Cooper and Patrick Vlaskovits have created a short but 
excellent book called The Entrepreneurs Guide to Customer De- 
velopment, which provides a gentle introduction to the topic. 
You can buy it here: http://custdev.com or read my review here: 
http://www.startuplessonslearned.com/2010/07/entrepreneurs 
-guide-to-customer.html 


When I first began blogging about entrepreneurship, it was not 
nearly as common an occupation as it is now. Very few bloggers 
were actively working on new ideas about entrepreneurship, and 
together we debated and refined these ideas online. 

Dave McClure, founder of the venture firm 500 Startups, writes 
a blog at http://500hats.typepad.com/. 500 Startups has an excel- 
lent blog as well: http://blog.500startups.com/. Dave's “Startup 
Metrics for Pirates” presentation laid out a framework for think- 
ing about and measuring online services that greatly influenced 
the concept of “engines of growth.” You can see the original pre- 
sentation here: http://500hats.typepad.com/500blogs/2008/09/ 
startup-metri-2.html as well as my original reaction here: http:// 
www.startuplessonslearned.com/2008/09/three-drivers-of- 
growth-for-your.html 

Sean Ellis writes the Startup Marketing Blog, which has been 
influential in my thinking about how to integrate marketing 
into startups: http://startup-marketing.com/ 

Andrew Chen's blog Futuristic Play is one of the best sources 
for thoughts on viral marketing, startup metrics, and design: 
http://andrewchenblog.com/ 
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Babak Nivi writes the excellent blog Venture Hacks and was 
an early Lean Startup evangelist: http://venturehacks.com/. He’s 
since gone on to create Angel List, which matches startups and 
investors around the world: http://angel.co/ 


Other fantastic Lean Startup blogs include: 


¢ Ash Maurya has emerged as a leader in helping boot- 
strapped online businesses apply Lean Startup ideas. His 
blog is called Running Lean, and he also has released an 
eBook of the same name. Both can be found here: http:// 
www.runningleanhq.com/ 

¢ Sean Murphy on early-stage software startups: http:// 
www.skmurphy.com/blog/ 

© Brant Coopers Market by Numbers: http://market-by- 

numbers.com/ 

¢ Patrick Vlaskovits on technology, customer development, 
and pricing: http://vlaskovits.com/ 

¢ The KISSmetrics Marketing Blog: http://blog.kissmetrics 
.com/ and Hiten Shah’s http://hitenism.com 


FURTHER READING 


Clayton M. Christensen’s The Innovators Dilemma and The In- 
novators Solution are classics. In addition, Christensen’s more 
recent work is also extremely helpful for seeing the theory of dis- 
ruptive innovation in practice, including The Innovator’ Prescrip- 
tion (about disrupting health care) and Disrupting Class (about 
education). 

http://ericri.es/ClaytonChristensen 


Geoffrey A. Moore's early work is famous among all entrepre- 
neurs, especially Crossing the Chasm and Inside the Tornado. But 
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he has continued to refine his thinking, and I have found his 
latest work, Dealing with Darwin: How Great Companies In- 
novate at Every Phase of Their Evolution, especially useful. 
http://ericri.es/DealingWithDarwin 


The Principles of Product Development Flow: Second Generation 
Lean Product Development by Donald G. Reinertsen. 
hetp://ericri.es/pdflow 


The Toyota Way by Jeffrey Liker. 
hetp://ericri.es/thetoyotaway 


Lean Thinking: Banish Waste and Create Wealth in Your 
Corporation, Revised and Updated by James P. Womack and 
Daniel T: Jones. 

hetp://ericri.es/LeanThinking 


The Peoples Tycoon: Henry Ford and the American Century 
by Steven Watts. 
hetp://ericri.es/ThePeoplesTycoon 


The One Best Way: Frederick Winslow Taylor and the Enigma of 
Efficiency by Robert Kanigel. 
http://ericri.es/OneBest Way 


The Principles of Scientific Management 
by Frederick Winslow Taylor. 
hetp://ericri.es/ScientificManagement 


Extreme Programming Explained: Embrace Change 
by Kent Beck and Cynthia Andres. 
hetp://ericri.es/EmbraceChange 
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Toyota Production System: Beyond Large-Scale Production by 
‘Taitichi Ohno. 
http://ericri.es/TaiichiOhno 


The idea of the Build-Measure-Learn feedback loop owes a lot 
to ideas from maneuver warfare, especially John Boyd’s OODA 
(Observe-Orient-Decide-Act) Loop. The most accessible intro- 
duction to Boyd’s ideas is Certain to Win: The Strategy of. ‘John 
Boyd, Applied to Business by Chet Richards. 
http://ericri.es/CertainToWin 


Out of the Crisis by W. Edwards Deming. 
hetp://ericri.es/OutOf TheCrisis 


My Years with General Motors by Alfred Sloan. 
hetp://ericri.es/MyYears 


Billy, Alfred, and General Motors: The Story of Two Unique Men, 
a Legendary Company, and a Remarkable Time in American 
History by William Pelfrey. 

hetp://ericri.es/BillyAlfred 


The Practice of Management by Peter F. Drucker. 
http://ericri.es/PracticeOfManagement 


Getting to Plan B: Breaking Through to a Better Business Model 
by John Mullins and Randy Komisar. 
hetp://ericri.es/Getting [oPlanB 


Endnotes 


Introduction 


1. For an up-to-date listing of Lean Startup meetups or to find one 
near you, see http://lean-startup.meetup.com or the Lean Startup 
Wiki: htep://leanstartup.pbworks.com/Meetups. See also Chapter 
14, “Join the Movement.” | 


Chapter 1. Start 


1. Manufacturing statistics and analysis are drawn from the blog 
Five Thirty Eight: http://www.fivethirtyeight.com/2010/02/us- 


manufacturing-is-not-dead.html 


Chapter 2. Define 


1. The Innovator’ Dilemma is a classic text by Clayton Christensen 
about the difficulty established companies have with disruptive 
innovation. Along with its sequel, The Innovator’ Solution, it 
lays out specific suggestions for how established companies can 
create autonomous divisions to pursue startup-like innovation. 
These specific structural prerequisites are discussed in detail in 
Chapter 12. 

2. For more about SnapTax, see http://blog.turbotax.intuit.com/ 
turbotax-press-releases/taxes-on-your-mobile-phone-it% 
E2%80%99s-a-snap/01142011-4865 and _http://mobilized.all 
thingsd.com/20110204/exclusive-intuit-sees-more-than- 
350000-downloads-for-snaptax-its-smartphone-tax-filing-app/ 

3. Most information relating to Intuit and SnapTax comes from 
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private interviews with Intuit management and employees. In- 
formation about Intuit’s founding comes from Suzanne Taylor 
and Kathy Schroeder’s [Inside Intuit: How the Makers of Quicken 
Beat Microsoft and Revolutionized an Entire Industry (Cambridge, 
Mass.: Harvard Business Press, 2003). 


Chapter 3. Learn 


l. 


Zz. 


The original five founders of IMVU were Will Harvey, Marcus 
Gosling, Matt Danzig, Mel Guymon, and myself. 

Usage in the United States was even more concentrated; see hrtp:// 
www.businessweek.com/technology/tech_stats/im050307.htm 

To hear moreabout IMVU’searly conversations with customers that 
led to our pivot away from the add-on strategy, see:’http://mixergy 
.com/ries-lean/ | 

A word of caution: demonstrating validated learning requires 
the right kind of metrics, called actionable metrics, which are dis- 
cussed in Chapter 7. 

This case was written by Bethany Coates under the direction of 
Professor Andy Rachleff. You can get a copy here: http://hbr.org/ 
product/imvu/an/E254-PDF-ENG 


Chapter 4. Experiment 


L 


Some entrepreneurs have adopted this slogan as their startup phi- 
losophy, using the acronym JFDI. A recent example can be seen 
at http://www.cloudave.com/1171/what-makes-an-entrepreneur- 
four-letters-jfdi/ 
http://techcrunch.com/2009/11/02/amazon-closes-zappos-deal- 
ends-up-paying-1—2-billion/ 

I want to thank Caroline Barlerin and HP for allowing me to 
include my experimental analysis of this new project. 
Information about Kodak Gallery comes from interviews con- 
ducted by Sara Leslie. 

The VLS story was recounted by Elnor Rozenrot, formerly of 
Innosight Ventures. Additional detail was provided by Akshay 
Mehra. For more on the VLS, see the article in Harvard Business 
Review: http://hbr.org/2011/01/new-business-models-in-emerging 
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-markets/ar/1 or press coverage at http://economictimes.indiatimes 
.com/news/news-by-company/corporate-trends/village-laundry 
-services-takes-on-the-dhobi/articleshow/5325032.cms 

For more on the early efforts of the CFPB, see the Wall Street 
Journal’s April 13, 2011, article “For Complaints, Don’t Call 
Consumer Bureau Yet”; http://online.wsj.com/article/SB100014 
2405274870355 1304576260772357440148.html. Many dedi- 
cated public servants are currently working hard to incorporate 
this experimental approach in the public sector under the leader- 
ship of President Obama. I would like to thank Aneesh Chopra, 
Chris Vein, Todd Park, and David Forrest for introducing me to 
these groundbreaking efforts. 


Chapter 5. Leap 


1. 


For example, CU Community, which began at Columbia Uni- 
versity, had an early head start. See http://www.slate.com/id/ 
2269131/. This account of Facebook's founding is drawn from 
David Kirkpatrick's The Facebook Effect (New York: Simon & 
Schuster, 2011). 

Actual engagement numbers from 2004 are hard to find, but 
this pattern has been consistent throughout Facebook’s pub- 
lic statements. For example, Chris Hughes reported in 2005 
that “60% log in daily. About 85% log in at least once a week, 
and 93% log in at least once a month.” http://techcrunch.com/ 
2005/09/07/85-of-college-students-use-facebook/ 

I first heard the term /eap of faith applied to startup assumptions by 
Randy Komisar, a former colleague and current partner at the ven- 
ture firm Kleiner Perkins Caufield & Byers. He expands on the con- 
cept in his book Gezting to Plan B, coauthored with John Mullins. 
http://www. forbes.com/2009/09/17/venture-capital-ipod- 
intelligent-technology-komisar.html 

‘A carefully researched table compiled for Motor magazine by 
Charles E. Duryea, himself a pioneer carmaker, revealed that 
from 1900 to 1908, 501 companies were formed in the United 
States for the purpose of manufacturing automobiles. Sixty per- 
cent of them folded outright within a couple of years; another 
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a 


10. 


l. 
2. 


6 percent moved into other areas of production.” This quote is 

from the Ford biography The People’s Tycoon: Henry Ford and the 
American Century by Steven Watts (New York: Vintage, 2006). . 
Jeffrey K. Liker, The Toyota Way. New York: McGraw-Hill, 2003, 

p. 223. 

http://www.autofieldguide.com/articles/030302.html 

In the customer development model, this is called customer discovery. 

For more on the founding of Intuit, see Suzanne Taylor and Kathy 

Schroeder, Inside Intuit. | 

For moreon the Lean UX movement, see http://www.cooper.com/ 

journal/2011/02/lean_ux_product_stewardship_an.html and 

http://www.slideshare.net/jgothelflean-ux-getting-out-of-the 

-deliverables-business 


Chapter 6. Test 


http://www.pluged.in/groupon-story-297/ 
“Groupon’s $6 Billion Gambler,” Wéall Street Journal; hetp:// 
online.wsj.com/article_email/SB1000142405274870482810 
4576021481410635432-IMyQjAxMTAwMDEwODExNDgy 
Wj.html 
The term minimum viable product has been in use since at least 
2000 as part of various approaches to product development. For 
an academic example, see http://www2.cs.uidaho.edu/-billjunk/ 
Publications/DynamicBalance.pdf | 

See also Frank Robinson of PMDI, who refers to a version 
of the product that is the smallest needed to sell to potential cus- 
tomers (http://productdevelopment.com/howitworks/mvp.html). 
This is similar to Steve Blank’s concept of the “minimum feature 
set” in customer development (http://steveblank.com/2010/03/04/ 
perfection-by-subtraction-the-minimum-feature-set/). My use of 
the term here has been generalized to any version of a product that can 
begin the process of learning, using the Build-Measure-Learn feed- 
back loop. For more, see http://www.startuplessonslearned.com/ 
2009/08/minimum-viable-product-guide.html 
Many people have written about this phenomenon, using varying 
terminology. Probably the most widely read is Geoffrey Moore's 
Crossing the Chasm. For more, see Eric Von Hippel’s research into 


8. 


9. 
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what he termed “lead users”; his book ‘The Sources of Innovation 
is a great place to start. Steve Blank uses the term earlyvangelist to 
emphasize the evangelical powers of these early customers. 

“To the casual observer, the Dropbox demo video looked like 
a normal product demonstration,” Drew says, “but we put in 
about a dozen Easter eggs that were tailored for the Digg audi- 
ence. References to Tay Zonday and ‘Chocolate Rain’ and al- 
lusions to Office Space and XKCD. It was a tongue-in-cheek 
nod to that crowd, and it kicked off a chain reaction. Within 24 
hours, the video had more than 10,000 Diggs.” http://answers 
-oreilly.com/topic/1372-marketing-lessons-from-dropbox-a-qa 
-with-ceo-drew-houston/. You can see the original video as well 
as the reaction from the Digg community at http://digg.com/ 
software/Google_Drive_killer_comin g_from_MIT_Startup. 
For more on Dropbox’s success, see “Dropbox: The Hottest Startup 
You've Never Heard Of” at http://tech.fortune.cnn.com/2011/ 
03/16/cloud-computing-for-the-rest-of-us/ 

This description courtesy of Lifehacker: http://lifehacker.com/ 
5586203/food-on-the-table-builds-menus-and- Si 
-based-on-your-familys-preferences 

This list was compiled by my colleague, Professor Tom Eisen- 
mann at Harvard Business School, Launching Technology Ven- 
tures for a case that he authored on Aardvark for his new class. For 
more, see http://platformsandnetworks.blogspot.com/2011/01/ 
launching-tech-ventures-part-i-course.html 
http://www.robgo.org/post/568227990/product-leadership- 
series-user-driven-design-at 
http://venturebeat.com/2010/02/11/confirmed-google-buys- 
social-search-engine-aardvark-for-50-million/ 


10. This is the heart of the Innovator’ Dilemma by Clayton Christensen. 
11. For more, see http://bit.ly/DontLaunch 


Chapter 7. Measure 


l. 


2. 


By contrast, Google’s main competitor Overture (eventually 
bought by Yahoo) had a minimum account size of $50, which 
deterred us from signing up, as it was too expensive. 

For more details about Farb’s entrepreneurial journey, see 
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this Mixergy interview: http://mixergy.com/farbood-nivi-grockit 
-interview/ 


Chapter 8. Pivot (or Persevere) 


l. 


Zi 


http://www.slideshare.net/dbinetti/lean-startup-at-sxsw-votizen- 
pivot-case-study 

For more on Path, see http://techcrunch.com/2011/02/02/google- 
tried-to-buy-path-for-100-million-path-said-no/ and _http://tech 
crunch.com/2011/02/01kleiner-perkins-leads-8—5-million- 
round-for-path/ . 

Includes approximately $30 million of assets under management 
and approximately $150 million of assets under administration, 
as of April 1, 2011. 

For more on Wealthfront, see the case study written by Sarah Mil- 
stein at http://www.startuplessonslearned.com/2010/07/case-study- 
kaching-anatomy-of-pivot.html. For more on Wealthfront’s recent 
success, see http://bits.blogs.ntimes.com/2010/10/19/wealthfront- 
loses-the-sound-effects/ | 


: IMVU’s results have been shared publicly on a few occasions. 


For 2008, see http://www.worldsinmotion.biz/2008/06/imvu_ 
reaches_20_million_regist.php; for 2009 see http://www.imvu 
.com/about/press_releases/press_release_20091005_1.php, and for 
2010 see http://techcrunch.com/2010/04/24/imvu-revenue/ 
Business architecture isa concept explored in detail in Moore's Dealing 
with Darwin. “Organizational structure based on prioritizing one of 
two business models (Complexsystems model and Volume operations 
model). Innovation types are understood and executed in completely 
different ways depending on which model an enterprise adopts.” 
For more, see http://www.dealingwithdarwin.com/theBook/darwin 
Dictionary.php 


Chapter 9. Batch 


Lk; 


http://Issacademy.com/2008/03/24/a-response-to-the-video- 
skeptics/ 


10. 
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If youre having trouble accepting this fact, it really is helpful to 
watch it on video. One extremely detail-oriented blogger took one 
video and broke it down, second-by-second, to see where the time 
went: “You lose between 2 and 5 seconds every time you move the 
pile around between steps. Also, you have to manage the pile sev- 
eral times during a task, something you don’t have to do nearly 
as much with [single-piece flow]. This also has a factory corollary: 
storing, moving, retrieving, and looking for work in progress in- 
ventory.” See the rest of the commentary here: http://lssacademy. 
com/2008/03/24/a-response-to-the-video-skeptics/ 

Timothy Fitz, an early IMVU engineer, deserves credit for 
having coined the term continuous deployment in a blog post: 
http://timothyfitz.wordpress.com/2009/02/10/continuous 
-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/. 
The actual development of the continuous deployment system 
is the work of too many different engineers at IMVU for me to 
give adequate credit here. For details on how to get started with 
continuous deployment, see http://radar.oreilly.com/2009/03/ 
continuous-deployment-5-eas.html 

For technical details of Wealthfront’s continuous deployment 
setup, see http://eng.wealthfront.com/2010/05/deployment-infra 
structure-for.html and http://eng.wealthfront.com/2011/03/lean- 
startup-stage-at-sxsw. html 

This description of School of One was provided by Jennifer Caro- 
lan of NewSchools Venture Fund. 

For more on the large-batch death spiral, see The Principles of 
Product Development Flow: Second Generation Lean Product Devel- 
opment by Donald G. Reinertsen: http://bit.ly/pdflow 

These lean health care examples are courtesy of Mark Graban, 
author of Lean Hospitals (New York: Productivity Press, 2008). 
This illustrative story about pull is drawn from Lean Production 
Simplified by Pascal Dennis (New York: Productivity Press, 2007). 
For an example of this misunderstanding at work, see http://www 
.oreillygmt.eu/interview/fatboy-in-a-lean-world/ 

Information about Alphabet Energy comes from interviews con- 
ducted by Sara Leslie. 
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11. For more on Toyota's learning organization, see The Toyota Way by 


Jeffrey Liker. 


Chapter 10. Grow 


Ly 


The Hotmail story, along with many other examples, is recounted 
in Adam L. Penenberg’s Viral Loop. For more on Hotmail, also see 
http://www.fastcompany.com/magazine/27/neteffects.html 

For more on the four customer currencies of time, money, skill, 
and passion, see http://www.startuplessonslearned.com/2009/12/ 
business-ecology-and-four-customer.html 

http://pmarca-archive. posterous.com/the-pmarca-guide-to-start 
ups-part-4-the-only | 

This is the lesson of Geoffrey Moore’s bestselling book Crossing 
the Chasm (New York: Harper Paperbacks, 2002). 
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Toyota Production System: Beyond Large-Scale Production by Tai- 
ichi Ohno (New York: Productivity Press, 1988). 

For more on Net Promoter Score, see http://www.startuplesson- 
slearned.com/2008/1 1/net-promoter-score-operational-tool-to. 
html and The Ultimate Question by Fred Reichheld (Cambridge, 
Mass.: Harvard Business Press, 2006). 

Information about QuickBooks comes from interviews con- 
ducted by Marisa Porzig. 


Chapter 12. Innovate 


L, 


Jeffrey Liker, John E. Ettlie, and John Creighton Campbell, En- 
gineered in Japan: Japanese Technology-Management Practices (New 
York: Oxford University Press, 1995), p. 196. 

For one account, see PC Magazine’s “Looking Back: 15 Years of PC 
Magazine” by Michael Miller, http://www.pcmag.com/article2/ 
0,2817,35549,00.asp 

The following discussion owes a great deal to Geoffrey Moore's 
Dealing with Darwin (New York: Portfolio Trade, 2008). I have 
had success implementing this framework in companies of many 
different sizes. 
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Chapter 13. Epilogue: Waste Not 


http://www. ibiblio.org/eldritch/fwt/ti.html 
http://www.goodreads.com/author/quotes/66490.Peter_Drucker 
http://www. ibiblio.org/eldritch/fwt/ti.html 

In fact, some such research has already begun. For more on Lean 
Startup research programs, see Nathan Furr’s Lean Startup 
Research Project at BYU, http://nathanfurr.com/2010/09/15/ 
the-lean-startup-research-project/, and Tom Eisenmann of Har- 
vard Business School’s Launching Technology Ventures project, 
http://platformsandnetworks.blogspot.om/201 Au 01/launching 
-tech-ventures-part-iv.html - 
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